Contents | YDC | Documentation

curl -X POST https://ydc-index.io/v1/contents \
     -H "X-API-Key: <apiKey>" \
     -H "Content-Type: application/json" \
     -d '{
  "urls": [
    "https://en.wikipedia.org/wiki/Main_Page"
  ],
  "formats": [
    "html",
    "metadata"
  ]
}'

[
  {
    "url": "https://en.wikipedia.org/wiki/Main_Page",
    "title": "Wikipedia, the free encyclopedia",
    "html": "Wikipedia was just a dream.\ndiv class=“frb-subheader“>\n<span class=“frb-replaced“>December 2</span>: Readers <span class=“frb-replaced“>in the United States</span> deserve an explanation.\n</div>\n</div>\n<div class=“frb-message-content“>\n<p>\nPlease don't skip this 1-minute read. It's <span class=“frb-replaced“>Tuesday</span>, <span class=“frb-replaced“>December 2</span>, and if you're like us, you've used Wikipedia countless times. To settle an argument with a friend. To satisfy a curiosity. Whether it's 3 in the morning or afternoon, Wikipedia is useful in your life. Please give <span class=“frb-replaced“>$2.75</span>.\n</p>\n<p>\nWikipedia's been around since 2001. Back then, it was just a wildly ambitious, probably impossible dream. But it came together piece by piece—created by people, not machines. Wikipedia's not perfect, but it's always been free thanks to everyday readers.\n</p>\n<p>\nOnly 2% ever donate. But that small group makes a big difference. When you support Wikipedia, you're standing up for something simple",
    "metadata": {
      "site_name": "Wikipedia",
      "favicon_url": "https://api.ydc-index.io/favicon?domain=en.wikipedia.org&size=128"
    }
  }
]

Returns the HTML or Markdown of a target webpage.

Authentication

X-API-Keystring

A unique API Key is required to authorize API access. For details, see how to get your API Key.

Request

This endpoint expects an object.

urlslist of stringsOptional

Array of URLs to fetch the contents from.

formatslist of enumsOptional

Array of content formats to return. All included formats are returned in the response. Include “metadata” to get JSON-LD and OpenGraph information, if available.

Allowed values:

crawl_timeoutintegerOptional1-60Defaults to 10

Maximum time in seconds to wait for page content. Must be between 1 and 60 seconds. Default is 10 seconds.

Response

An array of JSON objects containing the page content of each web page

urlstring or nullformat: "uri"

The webpage URL whose content has been fetched.

titlestring or null

The title of the web page.

htmlstring or null

The retrieved HTML content of the web page.

markdownstring or null

The retrieved Markdown content of the web page.

metadataobject or null

Page metadata. Only returned when "metadata" is included in the formats array.

Authentication

Request

Response

Errors