{"id":8841,"date":"2024-08-20T22:07:24","date_gmt":"2024-08-21T05:07:24","guid":{"rendered":"https:\/\/jeremywhittaker.com\/?p=8841"},"modified":"2024-08-20T22:11:16","modified_gmt":"2024-08-21T05:11:16","slug":"accessing-polymarket-data-in-python","status":"publish","type":"post","link":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/","title":{"rendered":"Accessing Polymarket.com Data in Python"},"content":{"rendered":"\n<p>Recently I started exploring arbitrage opportunities in the <a href=\"https:\/\/polymarket.com\/\">Polymarket.com<\/a> world. I quickly realized I would need faster access to the data. So I wrote some Python programs to access the platform for data analytics. The documents for doing this are located in the <a href=\"https:\/\/docs.polymarket.com\/#introduction\">Polymarket docs<\/a>. Here&#8217;s how it works. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Install py-clob-client<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">pip install py-clob-client&lt;br>&lt;br><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Coinbase Wallet<\/h2>\n\n\n\n<p>To perform this first block of code you need a Coinbase wallet. Once you have it setup you need to export your private key. <\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">from py_clob_client.client import ClobClient\n\nhost = \"https:\/\/clob.polymarket.com\"\nprivate_key = \"YOUR_PRIVATE_KEY_GOES_HERE\"\nchain_id = 137  # Polygon Mainnet\n\n# Initialize the client with private key\nclient = ClobClient(host, key=private_key, chain_id=chain_id)\n\napi_key_data = client.create_api_key()\n\nprint(api_key_data)<\/pre>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Creating keys.py<\/h2>\n\n\n\n<p>We can take this outputted data and create our keys.py which will hold our keys.For some reason, the <em>api_passphrase <\/em>and rename it <em>api_key <\/em>while commenting out the original <em>api_key<\/em>. I left the <em>api_secret <\/em>and original <em>api_passphrase<\/em>. Now that I have my keys.py I can reference it in my main code<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><br>Extracting all of the data<\/h2>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">import csv\nimport json\nfrom py_clob_client.client import ClobClient\nfrom keys import api_key  # Import only the API key\nfrom py_clob_client.clob_types import OpenOrderParams\n\n# Replace with your actual host and chain ID\nhost = \"https:\/\/clob.polymarket.com\"\nchain_id = 137  # Polygon Mainnet\n\n# Initialize the client with only the host, key, and chain_id\nclient = ClobClient(\n    host,\n    key=api_key,\n    chain_id=chain_id\n)\n\n# Initialize variables for pagination\nmarkets_list = []\nnext_cursor = None\n\n# Fetch all available markets using pagination\nwhile True:\n    try:\n        # Print the cursor value for debugging\n        print(f\"Fetching markets with next_cursor: {next_cursor}\")\n\n        # Make the API call based on the cursor value\n        if next_cursor is None:\n            response = client.get_markets()\n        else:\n            response = client.get_markets(next_cursor=next_cursor)\n\n        # Print the raw response for debugging\n        print(f\"API Response: {json.dumps(response, indent=2)}\")\n\n        # Check if the response is successful and contains data\n        if 'data' not in response:\n            print(\"No data found in response.\")\n            break\n\n        markets_list.extend(response['data'])\n        next_cursor = response.get(\"next_cursor\")\n\n        # Exit loop if there's no next_cursor indicating no more data to fetch\n        if not next_cursor:\n            break\n\n    except Exception as e:\n        # Print the exception details for debugging\n        print(f\"Exception occurred: {e}\")\n        print(f\"Exception details: {e.__class__.__name__}\")\n        print(f\"Error message: {e.args}\")\n        break\n\n# Debugging step: Print out the raw data to understand its structure\nprint(\"Raw Market Data:\")\nprint(json.dumps(markets_list, indent=2))\n\n# Dynamically extract all keys from the markets to create the CSV columns\ncsv_columns = set()\nfor market in markets_list:\n    csv_columns.update(market.keys())\n    # Also include nested keys like tokens\n    if 'tokens' in market:\n        csv_columns.update({f\"token_{key}\" for token in market['tokens'] for key in token.keys()})\n\ncsv_columns = sorted(csv_columns)  # Sort columns alphabetically for consistency\n\n# Writing to CSV\ncsv_file = \"markets_data.csv\"\ntry:\n    with open(csv_file, 'w', newline='') as csvfile:\n        writer = csv.DictWriter(csvfile, fieldnames=csv_columns)\n        writer.writeheader()\n        for market in markets_list:\n            row = {}\n            for key in csv_columns:\n                # Handling nested 'tokens' structure\n                if key.startswith(\"token_\"):\n                    token_key = key[len(\"token_\"):]\n                    row[key] = ', '.join([str(token.get(token_key, 'N\/A')) for token in market.get('tokens', [])])\n                else:\n                    row[key] = market.get(key, 'N\/A')\n            writer.writerow(row)\n    print(f\"Data has been written to {csv_file} successfully.\")\nexcept IOError as e:\n    print(f\"Error writing to CSV: {e}\")\n<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Output CSV<\/h2>\n\n\n\n<p>If all goes as planned you will now have a giant CSV file named markets_data.csv with all the different markets and their values. You can now build on this code to manipulate it to your needs. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recently I started exploring arbitrage opportunities in the Polymarket.com world. I quickly realized I would need faster access to the data. So I wrote some Python programs to&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-8841","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Accessing Polymarket.com Data in Python - Jeremy Whittaker<\/title>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Accessing Polymarket.com Data in Python - Jeremy Whittaker\" \/>\n<meta property=\"og:description\" content=\"Recently I started exploring arbitrage opportunities in the Polymarket.com world. I quickly realized I would need faster access to the data. So I wrote some Python programs to...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\" \/>\n<meta property=\"og:site_name\" content=\"Jeremy Whittaker\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/WhittakerJeremy\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/WhittakerJeremy\" \/>\n<meta property=\"article:published_time\" content=\"2024-08-21T05:07:24+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-08-21T05:11:16+00:00\" \/>\n<meta name=\"author\" content=\"JeremyWhittaker\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"JeremyWhittaker\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\"},\"author\":{\"name\":\"JeremyWhittaker\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"headline\":\"Accessing Polymarket.com Data in Python\",\"datePublished\":\"2024-08-21T05:07:24+00:00\",\"dateModified\":\"2024-08-21T05:11:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\"},\"wordCount\":191,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\",\"url\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\",\"name\":\"Accessing Polymarket.com Data in Python - Jeremy Whittaker\",\"isPartOf\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#website\"},\"datePublished\":\"2024-08-21T05:07:24+00:00\",\"dateModified\":\"2024-08-21T05:11:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/new.jeremywhittaker.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Accessing Polymarket.com Data in Python\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/new.jeremywhittaker.com\/#website\",\"url\":\"https:\/\/new.jeremywhittaker.com\/\",\"name\":\"Jeremy Whittaker\",\"description\":\"Research, software, markets, housing, and energy\",\"publisher\":{\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/new.jeremywhittaker.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c\",\"name\":\"JeremyWhittaker\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\",\"caption\":\"JeremyWhittaker\"},\"logo\":{\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g\"},\"sameAs\":[\"http:\/\/www.jeremywhittaker.com\",\"https:\/\/www.facebook.com\/WhittakerJeremy\",\"https:\/\/www.linkedin.com\/in\/jeremywhittaker\/\"],\"url\":\"https:\/\/new.jeremywhittaker.com\/index.php\/author\/jeremywhittaker\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Accessing Polymarket.com Data in Python - Jeremy Whittaker","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Accessing Polymarket.com Data in Python - Jeremy Whittaker","og_description":"Recently I started exploring arbitrage opportunities in the Polymarket.com world. I quickly realized I would need faster access to the data. So I wrote some Python programs to...","og_url":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/","og_site_name":"Jeremy Whittaker","article_publisher":"https:\/\/www.facebook.com\/WhittakerJeremy","article_author":"https:\/\/www.facebook.com\/WhittakerJeremy","article_published_time":"2024-08-21T05:07:24+00:00","article_modified_time":"2024-08-21T05:11:16+00:00","author":"JeremyWhittaker","twitter_card":"summary_large_image","twitter_misc":{"Written by":"JeremyWhittaker","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#article","isPartOf":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/"},"author":{"name":"JeremyWhittaker","@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"headline":"Accessing Polymarket.com Data in Python","datePublished":"2024-08-21T05:07:24+00:00","dateModified":"2024-08-21T05:11:16+00:00","mainEntityOfPage":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/"},"wordCount":191,"commentCount":0,"publisher":{"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/","url":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/","name":"Accessing Polymarket.com Data in Python - Jeremy Whittaker","isPartOf":{"@id":"https:\/\/new.jeremywhittaker.com\/#website"},"datePublished":"2024-08-21T05:07:24+00:00","dateModified":"2024-08-21T05:11:16+00:00","breadcrumb":{"@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/new.jeremywhittaker.com\/index.php\/2024\/08\/20\/accessing-polymarket-data-in-python\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/new.jeremywhittaker.com\/"},{"@type":"ListItem","position":2,"name":"Accessing Polymarket.com Data in Python"}]},{"@type":"WebSite","@id":"https:\/\/new.jeremywhittaker.com\/#website","url":"https:\/\/new.jeremywhittaker.com\/","name":"Jeremy Whittaker","description":"Research, software, markets, housing, and energy","publisher":{"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/new.jeremywhittaker.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/new.jeremywhittaker.com\/#\/schema\/person\/ed0edfdefb3e180693efef453372980c","name":"JeremyWhittaker","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g","caption":"JeremyWhittaker"},"logo":{"@id":"https:\/\/secure.gravatar.com\/avatar\/c8ac20e6dfa86b5f27ce9bffee4851099770cbea5ae7338a274865bfbc8c0218?s=96&d=retro&r=g"},"sameAs":["http:\/\/www.jeremywhittaker.com","https:\/\/www.facebook.com\/WhittakerJeremy","https:\/\/www.linkedin.com\/in\/jeremywhittaker\/"],"url":"https:\/\/new.jeremywhittaker.com\/index.php\/author\/jeremywhittaker\/"}]}},"_links":{"self":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts\/8841","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/comments?post=8841"}],"version-history":[{"count":7,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts\/8841\/revisions"}],"predecessor-version":[{"id":8851,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/posts\/8841\/revisions\/8851"}],"wp:attachment":[{"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/media?parent=8841"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/categories?post=8841"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/new.jeremywhittaker.com\/index.php\/wp-json\/wp\/v2\/tags?post=8841"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}