{"id":160,"date":"2016-08-19T09:01:46","date_gmt":"2016-08-19T09:01:46","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=160"},"modified":"2016-08-19T09:03:23","modified_gmt":"2016-08-19T09:03:23","slug":"php-curl-wrappers-to-consume-apis-crawl-and-more","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/","title":{"rendered":"PHP cURL wrappers to consume APIs, crawl and more&#8230;"},"content":{"rendered":"<p>cURL might be confusing for many, but it is matter of learning the basics on how to config the options, knowing which method (GET, POST, PUT) but..<\/p>\n<p><strong>Why in the hell would you use curl for?\u00a0<\/strong><\/p>\n<ul>\n<li>Requests to API&#8217;s sending parameters and stuff. Consume them easily.<\/li>\n<li>Requests to websites, download and stuff.<\/li>\n<li>Crawl websites parsing DOM and following links.<\/li>\n<li>many more..<\/li>\n<\/ul>\n<pre class=\"lang:php decode:true \">\/\/ the best fuction so you stop saving time.\r\nfunction curlwiz($uri, $method='GET', $data=null, $curl_headers=array(), $curl_options=array()) {\r\n  \/\/ default curl options which will almost be static, you can modify if you want\r\n  $default_curl_options = array(\r\n    CURLOPT_SSL_VERIFYPEER =&gt; false,\r\n    CURLOPT_HEADER =&gt; true,\r\n    CURLOPT_RETURNTRANSFER =&gt; true,\r\n    CURLOPT_TIMEOUT =&gt; 3,\r\n  );\r\n  \/\/ you can set the default headers into this array, usually you dont need them.\r\n  $default_headers = array();\r\n\r\n  \/\/ We need to trim and change into MAYUS the method passed\r\n  $method = strtoupper(trim($method));\r\n  $allowed_methods = array('GET', 'POST', 'PUT', 'DELETE'); \/\/ array with allowed methods. \r\n\r\n  if(!in_array($method, $allowed_methods)) \/\/ if the method from input is not in allowed_methods array, then throw an error.\r\n    throw new \\Exception(\"'$method' is not valid cURL HTTP method.\");\r\n\r\n  if(!empty($data) &amp;&amp; !is_string($data))\r\n    throw new \\Exception(\"Invalid data for cURL request '$method $uri'\");\r\n\r\n  \/\/ init\r\n  $curl = curl_init($uri);\r\n\r\n  \/\/ apply default options\r\n  curl_setopt_array($curl, $default_curl_options);\r\n\r\n  \/\/ apply method specific options\r\n  switch($method) {\r\n    case 'GET':\r\n      break;\r\n    case 'POST':\r\n      if(!is_string($data))\r\n        throw new \\Exception(\"Invalid data for cURL request '$method $uri'\");\r\n      curl_setopt($curl, CURLOPT_POST, true);\r\n      curl_setopt($curl, CURLOPT_POSTFIELDS, $data);\r\n      break;\r\n    case 'PUT':\r\n      if(!is_string($data))\r\n        throw new \\Exception(\"Invalid data for cURL request '$method $uri'\");\r\n      curl_setopt($curl, CURLOPT_CUSTOMREQUEST, $method);\r\n      curl_setopt($curl, CURLOPT_POSTFIELDS, $data);\r\n      break;\r\n    case 'DELETE':\r\n      curl_setopt($curl, CURLOPT_CUSTOMREQUEST, $method);\r\n      break;\r\n  }\r\n\r\n  \/\/ apply user options\r\n  curl_setopt_array($curl, $curl_options);\r\n\r\n  \/\/ add headers\r\n  curl_setopt($curl, CURLOPT_HTTPHEADER, array_merge($default_headers, $curl_headers));\r\n\r\n  \/\/ parse result from curl\r\n  $raw = rtrim(curl_exec($curl));\r\n  \/\/var_dump($raw);\r\n  $lines = explode(\"\\r\\n\", $raw); \/\/ we exploder curl response line by line\r\n  var_dump($lines);\r\n  $headers = array(); \r\n  $content = '';\r\n  $write_content = false;\r\n  if(count($lines) &gt; 3) {\r\n    foreach($lines as $h) {\r\n      if($h == '')\r\n        $write_content = true;\r\n      else {\r\n        if($write_content)\r\n          $content .= $h.\"\\n\";\r\n        else\r\n          $headers[] = $h;\r\n      }\r\n    }\r\n  }\r\n  $error = curl_error($curl);\r\n\r\n  curl_close($curl);\r\n\r\n  \/\/ return\r\n  return array(\r\n    'raw' =&gt; $raw,\r\n    'headers' =&gt; $headers,\r\n    'content' =&gt; $content,\r\n    'error' =&gt; $error\r\n  );\r\n}\r\n\r\ncurlwiz('http:\/\/facebook.com', 'GET');<\/pre>\n<p>With this you will be able to easily do do cURL just using 1 line of code and pre-configured set up of cURL into a function and using switch in the function. So we can split to know which configuration needs to be done if GET, or which other with POST.<\/p>\n<p>There are many guides on how to config cURL but there is one better than all guides:<\/p>\n<p><a href=\"http:\/\/php.net\/manual\/en\/curl.examples-basic.php\" target=\"_blank\">http:\/\/php.net\/manual\/en\/curl.examples-basic.php<\/a><\/p>\n<p>But there are also plenty of wrappers built by many other coders, but now you understand how they create so easy wrappers, many using OOP and many others with functions and switch (which I like much more).<\/p>\n<p>List of wrappers i found:<\/p>\n<ul>\n<li>cURLWrapper:\u00a0<a href=\"https:\/\/github.com\/svyatov\/CurlWrapper\" target=\"_blank\">https:\/\/github.com\/svyatov\/CurlWrapper<\/a><\/li>\n<li>php-curl-class:\u00a0<a href=\"https:\/\/github.com\/php-curl-class\/php-curl-class\" target=\"_blank\">https:\/\/github.com\/php-curl-class\/php-curl-class<\/a><\/li>\n<li>php-curl:\u00a0<a href=\"https:\/\/github.com\/anlutro\/php-curl\" target=\"_blank\">https:\/\/github.com\/anlutro\/php-curl<\/a><\/li>\n<\/ul>\n<p>With that! you will be able to code like this lazy fat:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone\" src=\"https:\/\/media.giphy.com\/media\/QHE5gWI0QjqF2\/giphy.gif\" width=\"720\" height=\"480\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>cURL might be confusing for many, but it is matter of learning the basics on how to config the options, knowing which method (GET, POST, PUT) but.. Why in the hell would you use curl for?\u00a0 Requests to API&#8217;s sending parameters and stuff. Consume them easily. Requests to websites, download and stuff. Crawl websites parsing [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":161,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[10,6],"tags":[87,14,89,88],"class_list":["post-160","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-php","category-tutorials","tag-curls","tag-php","tag-resources","tag-tutorial"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots\" \/>\n<meta property=\"og:description\" content=\"cURL might be confusing for many, but it is matter of learning the basics on how to config the options, knowing which method (GET, POST, PUT) but.. Why in the hell would you use curl for?\u00a0 Requests to API&#8217;s sending parameters and stuff. Consume them easily. Requests to websites, download and stuff. Crawl websites parsing [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/\" \/>\n<meta property=\"og:site_name\" content=\"Wizard Of Bots\" \/>\n<meta property=\"article:published_time\" content=\"2016-08-19T09:01:46+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-08-19T09:03:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"500\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"wizardofbots\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"wizardofbots\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/\",\"url\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/\",\"name\":\"PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots\",\"isPartOf\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage\"},\"thumbnailUrl\":\"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg\",\"datePublished\":\"2016-08-19T09:01:46+00:00\",\"dateModified\":\"2016-08-19T09:03:23+00:00\",\"author\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\"},\"breadcrumb\":{\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage\",\"url\":\"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg\",\"contentUrl\":\"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg\",\"width\":800,\"height\":500},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/wizardofbots.com\/network\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"PHP cURL wrappers to consume APIs, crawl and more&#8230;\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#website\",\"url\":\"http:\/\/wizardofbots.com\/network\/\",\"name\":\"Wizard Of Bots\",\"description\":\"Botting and AI community\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/wizardofbots.com\/network\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\",\"name\":\"wizardofbots\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"caption\":\"wizardofbots\"},\"url\":\"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/","og_locale":"en_US","og_type":"article","og_title":"PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots","og_description":"cURL might be confusing for many, but it is matter of learning the basics on how to config the options, knowing which method (GET, POST, PUT) but.. Why in the hell would you use curl for?\u00a0 Requests to API&#8217;s sending parameters and stuff. Consume them easily. Requests to websites, download and stuff. Crawl websites parsing [&hellip;]","og_url":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/","og_site_name":"Wizard Of Bots","article_published_time":"2016-08-19T09:01:46+00:00","article_modified_time":"2016-08-19T09:03:23+00:00","og_image":[{"width":800,"height":500,"url":"https:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg","type":"image\/jpeg"}],"author":"wizardofbots","twitter_card":"summary_large_image","twitter_misc":{"Written by":"wizardofbots","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/","url":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/","name":"PHP cURL wrappers to consume APIs, crawl and more... - Wizard Of Bots","isPartOf":{"@id":"http:\/\/wizardofbots.com\/network\/#website"},"primaryImageOfPage":{"@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage"},"image":{"@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage"},"thumbnailUrl":"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg","datePublished":"2016-08-19T09:01:46+00:00","dateModified":"2016-08-19T09:03:23+00:00","author":{"@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f"},"breadcrumb":{"@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#primaryimage","url":"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg","contentUrl":"http:\/\/wizardofbots.com\/network\/wp-content\/uploads\/2016\/08\/curlwob.jpg","width":800,"height":500},{"@type":"BreadcrumbList","@id":"https:\/\/wizardofbots.com\/network\/php-curl-wrappers-to-consume-apis-crawl-and-more\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/wizardofbots.com\/network\/"},{"@type":"ListItem","position":2,"name":"PHP cURL wrappers to consume APIs, crawl and more&#8230;"}]},{"@type":"WebSite","@id":"http:\/\/wizardofbots.com\/network\/#website","url":"http:\/\/wizardofbots.com\/network\/","name":"Wizard Of Bots","description":"Botting and AI community","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/wizardofbots.com\/network\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f","name":"wizardofbots","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","caption":"wizardofbots"},"url":"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/"}]}},"_links":{"self":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/160"}],"collection":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=160"}],"version-history":[{"count":2,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/160\/revisions"}],"predecessor-version":[{"id":163,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/160\/revisions\/163"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media\/161"}],"wp:attachment":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=160"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=160"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=160"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}