{"id":176,"date":"2016-09-02T16:24:40","date_gmt":"2016-09-02T16:24:40","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=176"},"modified":"2016-09-02T16:24:40","modified_gmt":"2016-09-02T16:24:40","slug":"scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/scrape-with-nodejs-and-jquery-get-your-ip-host-and-user-agent\/","title":{"rendered":"Scrape with NodeJS and JQuery: Get your IP, Host and User-Agent"},"content":{"rendered":"<p>Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects.<\/p>\n<p>In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code:<\/p>\n<pre class=\"lang:js decode:true \">var request = require('request'); \/\/ we need request library\r\nvar cheerio = require('cheerio'); \/\/ and cheerio library\/ JQuery\r\n\/\/ set some defaults\r\nreq = request.defaults({\r\n  jar: true,                 \/\/ save cookies to jar\r\n  rejectUnauthorized: false, \r\n  followAllRedirects: true   \/\/ allow redirections\r\n});\r\n\/\/ scrape the page\r\nreq.get({\r\n    url: \"http:\/\/www.whatsmyip.org\/\",\r\n    headers: {\r\n        'User-Agent': 'Google' \/\/ You can put the user-agent that you want\r\n     }\r\n  }, function(err, resp, body) {\r\n  \r\n  \/\/ load the html into cheerio\r\n  var $ = cheerio.load(body);\r\n  \r\n  \/\/ get the data and output to console\r\n  console.log( 'IP: ' + $('#ip').text() );  \/\/scrape using CSS selector\r\n  console.log( 'Host: ' + $('#hostname').text() );\r\n  console.log( 'User-Agent: ' + $('#useragent').text() );\r\n});<\/pre>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hey fellas, sorry for being absent for a long time, mainly it was lots of work on other projects. In this post I am going to teach you how to screen scrape using NodeJS and JQuery (cheerio). Its relatively easy, here is the code: var request = require(&#8216;request&#8217;); \/\/ we need request library var cheerio [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[33],"tags":[100,39,99,11,88],"class_list":["post-176","post","type-post","status-publish","format-standard","hentry","category-javascript","tag-cheerio","tag-nodejs","tag-request","tag-scraping","tag-tutorial"],"_links":{"self":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=176"}],"version-history":[{"count":1,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176\/revisions"}],"predecessor-version":[{"id":177,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/176\/revisions\/177"}],"wp:attachment":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=176"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=176"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=176"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}