Diffbot Knowledge Graph Search

Diffbot Knowledge Graph Search (DQL)¶

Bridge402 provides access to Diffbot's Knowledge Graph Search API (DQL) through x402-payment-protected endpoints. Search the Diffbot Knowledge Graph for organizations, people, articles, and more using DQL (Diffbot Query Language) queries with crypto payments.

Overview¶

The Diffbot Knowledge Graph Search API allows you to query a massive knowledge graph containing billions of facts about organizations, people, articles, products, and more. Instead of managing API keys or subscriptions, pay per search with crypto.

Primary Use Cases: - Company Research: Find organizations by location, size, industry, and other attributes - People Discovery: Search for individuals by role, company, location, or other criteria - Content Discovery: Find articles, products, or discussions matching specific criteria - Data Analysis: Query structured data from the knowledge graph for analytics - AI Agent Research: Enable AI agents to search and discover information programmatically

What is DQL?¶

DQL (Diffbot Query Language) is a query language designed for searching the Diffbot Knowledge Graph. It's similar to SQL but optimized for graph data.

Example DQL Query:

type:Organization locations.city.name:"San Francisco" nbEmployees>5000

This query searches for all Organizations located in San Francisco with more than 5000 employees.

API Endpoint¶

Search Knowledge Graph¶

GET /diffbot/search

Search the Diffbot Knowledge Graph using DQL (Diffbot Query Language).

Query Parameters:

Parameter	Type	Required	Description	Example
`query`	string	Yes	DQL query string	`type:Organization locations.city.name:"San Francisco" nbEmployees>5000`
`type`	string	No	Query type (`query`, `text`, `queryTextFallback`, `crawl`)	`query`
`col`	string	No	Comma-delimited list of custom crawl collections	`collection1,collection2`
`from`	integer	No	Return results starting at this index (0-based)	`0`
`size`	integer	No	Maximum number of results to return (-1 for all)	`50`
`format`	string	No	Output format (`json`, `jsonl`, `csv`, `xls`, `xlsx`)	`json`
`exportspec`	string	No	Export spec defining columns to export	`name;summary`
`exportseparator`	string	No	Export separator for CSV/Excel formats	`,`
`exportfile`	string	No	Export file name	`results.csv`
`filter`	string	No	Semi-colon separated path filter (JsonPath)	`$.name;$.locations.country.name`
`jsonmode`	string	No	JSON mode (`extended` or `id`)	`extended`
`nonCanonicalFacts`	boolean	No	Return non-canonical facts	`false`
`noDedupArticles`	boolean	No	Do not deduplicate articles	`false`
`cluster`	string	No	Clustering mode (`all`, `best`, `dedupe`)	`best`
`report`	boolean	No	Generate coverage report	`false`
`network`	string	No	Payment network preference	`base` or `sol`/`solana`

Headers:

Header	Type	Required	Description
`X-PAYMENT`	string	Yes*	Base64-encoded x402 payment data

*Required for access. If omitted, returns payment invoice (402 response).

DQL Query Syntax¶

Basic Queries¶

Search by Type:

type:Organization

Search with Filters:

type:Organization locations.city.name:"San Francisco"

Search with Comparisons:

type:Organization nbEmployees>5000

Combined Queries:

type:Organization locations.city.name:"San Francisco" nbEmployees>5000

Common Query Patterns¶

Find Organizations in a City:

type:Organization locations.city.name:"New York"

Find Organizations by Employee Count:

type:Organization nbEmployees>1000 nbEmployees<10000

Find People by Role:

type:Person roles.role:"CEO"

Find Articles by Date:

type:Article date>2024-01-01

Find Products by Price:

type:Product offerPrice<100

Request Examples¶

Get Payment Invoice (Without Payment)¶

curl "https://bridge402.tech/diffbot/search?query=type:Organization%20locations.city.name:%22San%20Francisco%22%20nbEmployees%3E5000&network=sol"

Response (402 Payment Required):

{
  "x402Version": 1,
  "error": "X-PAYMENT header is required",
  "accepts": [
    {
      "scheme": "exact",
      "network": "solana",
      "maxAmountRequired": "10000",
      "asset": "EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v",
      "payTo": "BjxbJg48jQmoBLJnRunB1CMY5SZwvcUmnXCaWNeSXBei",
      "resource": "https://bridge402.tech/diffbot/search",
      "description": "Diffbot Knowledge Graph search (DQL) [Solana/USDC]",
      "mimeType": "application/json",
      "maxTimeoutSeconds": 120,
      "extra": {
        "product": "Bridge402 Diffbot — Knowledge Graph Search (Solana)",
        "searchType": "dql",
        "query": "type:Organization locations.city.name:\"San Francisco\" nbEmployees>5000",
        "feePayer": "2wKupLR9q6wXYppw8Gr2NvWxKBUqm4PPJKkQfoxHDBg4"
      }
    }
  ],
  "searchType": "dql",
  "query": "type:Organization locations.city.name:\"San Francisco\" nbEmployees>5000"
}

Search with Payment¶

curl "https://bridge402.tech/diffbot/search?query=type:Organization%20locations.city.name:%22San%20Francisco%22%20nbEmployees%3E5000&size=10&network=sol" \
  -H "X-PAYMENT: <base64-encoded-x402-payment>"

Response (200 Success):

{
  "searchType": "dql",
  "query": "type:Organization locations.city.name:\"San Francisco\" nbEmployees>5000",
  "data": {
    "data": [
      {
        "name": "Company Name",
        "diffbotUri": "http://diffbot.com/entity/Company_Name",
        "type": "Organization",
        "locations": [
          {
            "city": {
              "name": "San Francisco"
            },
            "country": {
              "name": "United States"
            }
          }
        ],
        "nbEmployees": 7500,
        "summary": "Company description...",
        "homepage": "https://example.com"
      }
    ],
    "total": 1,
    "hits": 1
  },
  "payment": {
    "verified": true,
    "settled": true,
    "txHash": "5xK...",
    "network": "solana"
  },
  "metadata": {
    "provider": "Diffbot",
    "endpoint": "dql",
    "timestamp": 1703123456.789
  }
}

Response Format¶

Search Results¶

The data.data array contains the search results:

{
  "data": {
    "data": [
      {
        "name": "Entity Name",
        "diffbotUri": "http://diffbot.com/entity/Entity_Name",
        "type": "Organization",
        "locations": [...],
        "nbEmployees": 5000,
        "summary": "Entity description...",
        "homepage": "https://example.com"
      }
    ],
    "total": 1,
    "hits": 1
  }
}

Organization Result Example¶

{
  "name": "Apple Inc",
  "diffbotUri": "http://diffbot.com/entity/Apple_Inc",
  "type": "Organization",
  "locations": [
    {
      "city": {
        "name": "Cupertino"
      },
      "state": {
        "name": "California"
      },
      "country": {
        "name": "United States"
      }
    }
  ],
  "nbEmployees": 164000,
  "summary": "Apple Inc. is an American multinational technology company...",
  "homepage": "https://www.apple.com",
  "industries": [
    {
      "name": "Technology",
      "diffbotUri": "http://diffbot.com/entity/Technology"
    }
  ]
}

Person Result Example¶

{
  "name": "Tim Cook",
  "diffbotUri": "http://diffbot.com/entity/Tim_Cook",
  "type": "Person",
  "roles": [
    {
      "role": "CEO",
      "organization": {
        "name": "Apple Inc",
        "diffbotUri": "http://diffbot.com/entity/Apple_Inc"
      }
    }
  ],
  "locations": [
    {
      "city": {
        "name": "Cupertino"
      }
    }
  ]
}

Integration Examples¶

Python Example¶

import asyncio
import httpx
from urllib.parse import quote

async def search_knowledge_graph(dql_query: str, payment_data: str, size: int = 50):
    """Search Diffbot Knowledge Graph using DQL with x402 payment"""
    async with httpx.AsyncClient() as client:
        # URL encode the DQL query
        encoded_query = quote(dql_query)

        response = await client.get(
            f"https://bridge402.tech/diffbot/search",
            params={
                "query": dql_query,
                "size": size,
                "network": "sol"
            },
            headers={"X-PAYMENT": payment_data}
        )

        if response.status_code == 200:
            data = response.json()
            return data
        elif response.status_code == 402:
            # Payment required - get invoice
            invoice = response.json()
            print(f"Payment required: {invoice['accepts'][0]['maxAmountRequired']} atomic units")
            return invoice
        else:
            raise Exception(f"Request failed: {response.status_code} - {response.text}")

# Usage
query = 'type:Organization locations.city.name:"San Francisco" nbEmployees>5000'
result = await search_knowledge_graph(query, "<your-x402-payment>", size=10)

if result.get("data") and result["data"].get("data"):
    for org in result["data"]["data"]:
        print(f"Name: {org.get('name')}")
        print(f"Employees: {org.get('nbEmployees')}")
        print(f"Summary: {org.get('summary', '')[:100]}...")
        print("---")

JavaScript/Node.js Example¶

import { request } from 'undici';

async function searchKnowledgeGraph(dqlQuery, paymentData, options = {}) {
    const { size = 50, network = 'sol', ...otherParams } = options;

    // Build query string
    const params = new URLSearchParams({
        query: dqlQuery,
        size: size.toString(),
        network: network,
        ...otherParams
    });

    const url = `https://bridge402.tech/diffbot/search?${params.toString()}`;

    const res = await request(url, {
        method: 'GET',
        headers: {
            'X-PAYMENT': paymentData
        }
    });

    const data = await res.body.json();

    if (data.data && data.data.data) {
        return data.data.data.map(result => ({
            name: result.name,
            type: result.type,
            diffbotUri: result.diffbotUri,
            summary: result.summary,
            // Include other relevant fields
        }));
    }

    return data;
}

// Usage
const query = 'type:Organization locations.city.name:"San Francisco" nbEmployees>5000';
const results = await searchKnowledgeGraph(
    query,
    '<your-x402-payment>',
    { size: 10 }
);

results.forEach(org => {
    console.log(`Name: ${org.name}`);
    console.log(`Type: ${org.type}`);
    console.log(`Summary: ${org.summary?.substring(0, 100)}...`);
    console.log('---');
});

Advanced Query Example¶

# Find tech companies in San Francisco with 100-1000 employees
query = '''
type:Organization 
locations.city.name:"San Francisco" 
industries.name:"Technology" 
nbEmployees>100 
nbEmployees<1000
'''

result = await search_knowledge_graph(query, payment_data, size=20)

# Find CEOs of companies in New York
query = '''
type:Person 
roles.role:"CEO" 
roles.organization.locations.city.name:"New York"
'''

result = await search_knowledge_graph(query, payment_data, size=50)

Use Cases¶

Company Research¶

async def find_companies_in_city(city: str, min_employees: int = 100):
    """Find companies in a specific city with minimum employee count"""
    query = f'type:Organization locations.city.name:"{city}" nbEmployees>={min_employees}'
    result = await search_knowledge_graph(query, payment_data, size=100)

    companies = []
    if result.get("data") and result["data"].get("data"):
        for org in result["data"]["data"]:
            companies.append({
                "name": org.get("name"),
                "employees": org.get("nbEmployees"),
                "industry": org.get("industries", [{}])[0].get("name") if org.get("industries") else None,
                "homepage": org.get("homepage"),
                "summary": org.get("summary")
            })

    return companies

# Usage
sf_companies = await find_companies_in_city("San Francisco", min_employees=500)

People Discovery¶

async function findPeopleByRole(role, location = null) {
    let query = `type:Person roles.role:"${role}"`;

    if (location) {
        query += ` roles.organization.locations.city.name:"${location}"`;
    }

    const results = await searchKnowledgeGraph(query, paymentData, { size: 50 });

    return results.map(person => ({
        name: person.name,
        role: person.roles?.[0]?.role,
        company: person.roles?.[0]?.organization?.name,
        location: person.roles?.[0]?.organization?.locations?.[0]?.city?.name
    }));
}

// Usage
const ceos = await findPeopleByRole("CEO", "New York");

Content Discovery¶

async def find_recent_articles(topic: str, days: int = 7):
    """Find recent articles about a topic"""
    from datetime import datetime, timedelta

    cutoff_date = (datetime.now() - timedelta(days=days)).strftime("%Y-%m-%d")
    query = f'type:Article date>={cutoff_date} title:"{topic}"'

    result = await search_knowledge_graph(query, payment_data, size=50)

    articles = []
    if result.get("data") and result["data"].get("data"):
        for article in result["data"]["data"]:
            articles.append({
                "title": article.get("title"),
                "date": article.get("date"),
                "url": article.get("pageUrl"),
                "author": article.get("author"),
                "summary": article.get("summary")
            })

    return articles

# Usage
bitcoin_articles = await find_recent_articles("Bitcoin", days=7)

Error Handling¶

Common Errors¶

400 Bad Request

{
  "detail": "DQL query is required and must be a non-empty string"
}

402 Payment Required

{
  "x402Version": 1,
  "error": "X-PAYMENT header is required",
  "accepts": [...]
}

422 Unprocessable Entity - Invalid DQL query syntax - Check your query format and try again

500 Internal Server Error - Diffbot API may be unavailable - Retry the request

502 Bad Gateway - Upstream Diffbot API error - Verify Diffbot API key is configured on the server

Best Practices¶

Query Optimization: Use specific filters to reduce result set size and improve performance
Pagination: Use from and size parameters for large result sets
Field Filtering: Use filter parameter to return only needed fields
Error Handling: Always handle 402 responses to get payment requirements
Network Selection: Choose network based on your wallet capabilities (Base or Solana)
Query Caching: Cache results for identical queries to avoid redundant searches
URL Encoding: Always URL-encode DQL queries when using query parameters

Pricing¶

Cost: $0.01 USDC per search (10,000 atomic units)
Payment Networks: Base or Solana (USDC)
No Subscription Required: Pay-per-use model perfect for AI agents and intermittent access
All DQL Queries: Same price regardless of query complexity or result count

DQL Reference¶

For complete DQL syntax documentation, see: - Diffbot DQL Documentation - DQL Search API Reference

Support¶

For questions about Diffbot Knowledge Graph Search or integration help, refer to: - Payment Integration Guide - Diffbot Extraction - For URL-based content extraction - Diffbot NL Processing - For natural language processing - Contact the Bridge402 development team