Contents | YDC | Documentation

# Use our official Python SDK to fetch the contents of a web page
from youdotcom import You
from youdotcom.models import ContentsFormats
with You("api_key") as you:
    res = you.contents.generate(
        urls=[
            "https://en.wikipedia.org/wiki/Main_Page",
        ],
        formats=[ContentsFormats.HTML],
    )
    # Print the fetched HTML content
    for page in res:
        print(f"Title: {page.title}")
        print(f"HTML: {page.html[:500]}...")  # First 500 chars

[
  {
    "url": "https://en.wikipedia.org/wiki/Main_Page",
    "title": "Wikipedia, the free encyclopedia",
    "html": "Wikipedia was just a dream.\ndiv class=\"frb-subheader\">\n<span class=\"frb-replaced\">December 2</span>: Readers <span class=\"frb-replaced\">in the United States</span> deserve an explanation.\n</div>\n</div>\n<div class=\"frb-message-content\">\n<p>\nPlease don't skip this 1-minute read. It's <span class=\"frb-replaced\">Tuesday</span>, <span class=\"frb-replaced\">December 2</span>, and if you're like us, you've used Wikipedia countless times. To settle an argument with a friend. To satisfy a curiosity. Whether it's 3 in the morning or afternoon, Wikipedia is useful in your life. Please give <span class=\"frb-replaced\">$2.75</span>.\n</p>\n<p>\nWikipedia's been around since 2001. Back then, it was just a wildly ambitious, probably impossible dream. But it came together piece by piece—created by people, not machines. Wikipedia's not perfect, but it's always been free thanks to everyday readers.\n</p>\n<p>\nOnly 2% ever donate. But that small group makes a big difference. When you support Wikipedia, you're standing up for something simple",
    "metadata": {
      "site_name": "Wikipedia",
      "favicon_url": "https://api.ydc-index.io/favicon?domain=en.wikipedia.org&size=128"
    }
  }
]

Returns the HTML or Markdown of a target webpage.

Authentication

X-API-Keystring

The unique API Key required to authorize API access. Learn how to get yours in the “Get your API key” section of the documentation.

Request

This endpoint expects an object.

urlslist of stringsOptional

Array of URLs to fetch the contents from.

formatslist of enumsOptional

The formats of the content to be returned. Can include ‘html’, ‘markdown’, and/or ‘metadata’.

Allowed values:

crawl_timeoutdoubleOptional1-60

The timeout in seconds for crawling each URL. Must be between 1 and 60 seconds.

Response

An array of JSON objects containing the page content of each web page

urlstring or nullformat: "uri"

The webpage URL whose content has been fetched.

titlestring or null

The title of the web page.

htmlstring or null

The retrieved HTML content of the web page.

markdownstring or null

The retrieved Markdown content of the web page.

metadataobject or null

Metadata about the web page. Only returned when 'metadata' is included in the formats array.

Authentication

Request

Response

Errors