{"id":176,"date":"2016-09-02T16:24:40","date_gmt":"2016-09-02T16:24:40","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=176"},"modified":"2016-09-02T16:24:40","modified_gmt":"2016-09-02T16:24:40","slug":"scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","title":{"rendered":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent"},"content":{"rendered":"<p>Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects.<\/p>\n<p>In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code:<\/p>\n<pre class=\"lang:js decode:true \">var request = require('request'); \/\/ we need request library\r\nvar cheerio = require('cheerio'); \/\/ and cheerio library\/ JQuery\r\n\/\/ set some defaults\r\nreq = request.defaults({\r\n  jar: true,                 \/\/ save cookies to jar\r\n  rejectUnauthorized: false, \r\n  followAllRedirects: true   \/\/ allow redirections\r\n});\r\n\/\/ scrape the page\r\nreq.get({\r\n    url: \"http:\/\/www.whatsmyip.org\/\",\r\n    headers: {\r\n        'User-Agent': 'Google' \/\/ You can put the user-agent that you want\r\n     }\r\n  }, function(err, resp, body) {\r\n  \r\n  \/\/ load the html into cheerio\r\n  var $ = cheerio.load(body);\r\n  \r\n  \/\/ get the data and output to console\r\n  console.log( 'IP: ' + $('#ip').text() );  \/\/scrape using CSS selector\r\n  console.log( 'Host: ' + $('#hostname').text() );\r\n  console.log( 'User-Agent: ' + $('#useragent').text() );\r\n});<\/pre>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects. In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code: var request = require(&#8216;request&#8217;); \/\/ we need request library var cheerio [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[33],"tags":[100,39,99,11,88],"class_list":["post-176","post","type-post","status-publish","format-standard","hentry","category-javascript","tag-cheerio","tag-nodejs","tag-request","tag-scraping","tag-tutorial"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots\" \/>\n<meta property=\"og:description\" content=\"Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects. In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code: var request = require(&#039;request&#039;); \/\/ we need request library var cheerio [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/\" \/>\n<meta property=\"og:site_name\" content=\"Wizard Of Bots\" \/>\n<meta property=\"article:published_time\" content=\"2016-09-02T16:24:40+00:00\" \/>\n<meta name=\"author\" content=\"wizardofbots\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"wizardofbots\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/\",\"url\":\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/\",\"name\":\"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots\",\"isPartOf\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/#website\"},\"datePublished\":\"2016-09-02T16:24:40+00:00\",\"dateModified\":\"2016-09-02T16:24:40+00:00\",\"author\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\"},\"breadcrumb\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"http:\/\/wizardofbots.com\/network\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#website\",\"url\":\"http:\/\/wizardofbots.com\/network\/\",\"name\":\"Wizard Of Bots\",\"description\":\"Botting and AI community\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\/\/wizardofbots.com\/network\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\",\"name\":\"wizardofbots\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"caption\":\"wizardofbots\"},\"url\":\"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","og_locale":"en_US","og_type":"article","og_title":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots","og_description":"Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects. In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code: var request = require('request'); \/\/ we need request library var cheerio [&hellip;]","og_url":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","og_site_name":"Wizard Of Bots","article_published_time":"2016-09-02T16:24:40+00:00","author":"wizardofbots","twitter_card":"summary_large_image","twitter_misc":{"Written by":"wizardofbots","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","url":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","name":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent - Wizard Of Bots","isPartOf":{"@id":"http:\/\/wizardofbots.com\/network\/#website"},"datePublished":"2016-09-02T16:24:40+00:00","dateModified":"2016-09-02T16:24:40+00:00","author":{"@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f"},"breadcrumb":{"@id":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/"]}]},{"@type":"BreadcrumbList","@id":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"http:\/\/wizardofbots.com\/network\/"},{"@type":"ListItem","position":2,"name":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent"}]},{"@type":"WebSite","@id":"http:\/\/wizardofbots.com\/network\/#website","url":"http:\/\/wizardofbots.com\/network\/","name":"Wizard Of Bots","description":"Botting and AI community","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/wizardofbots.com\/network\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f","name":"wizardofbots","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","caption":"wizardofbots"},"url":"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/"}]}},"_links":{"self":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176"}],"collection":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=176"}],"version-history":[{"count":1,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176\/revisions"}],"predecessor-version":[{"id":177,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176\/revisions\/177"}],"wp:attachment":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=176"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=176"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=176"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}