{"id":284,"date":"2017-10-24T03:20:08","date_gmt":"2017-10-24T03:20:08","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=284"},"modified":"2017-10-25T01:33:40","modified_gmt":"2017-10-25T01:33:40","slug":"node-js-full-domain-crawler-email-extractor-source","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","title":{"rendered":"Node.JS full domain crawler + email extractor (source)"},"content":{"rendered":"<p>Whats up everybody!<\/p>\n<p>I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies.<\/p>\n<p>Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step by step:<\/p>\n<p>1.- Create a dir.<br \/>\n2.- Go inside of it.<br \/>\n3.- You need to install roboto and htmlstrip-native with npm.<br \/>\n4.- Create a crawl.js file inside that folder you created.<br \/>\n5.- Paste the source code on it:<\/p>\n\n\t\t<div class=\"subscribeunlock_container\">\n\t\t\t<div name=\"subscribeunlock\" class=\"subscribeunlock_box\" id=\"subscribeunlock_8042\">\n\t\t\t\t<div class=\"subscribeunlock_signup_form\" id=\"subscribeunlock_signup_form_8042\">\n\t\t\t\t\t<div class=\"subscribeunlock_form_row\"><p>Complete your name and email to unlock the code...<\/p><\/div>\n\t\t\t\t\t<div class=\"subscribeunlock_form_row\">\n\t\t\t\t\t\t\n\t\t\t\t\t\t<div class=\"subscribeunlock_form_column subscribeunlock_50\">\n\t\t\t\t\t\t\t<div>\n\t\t\t\t\t\t\t\t<input required=\"required\" tabindex=\"101\" class=\"subscribeunlock-input subscribeunlock_fullwidth\" type=\"text\" id=\"name_8042\" placeholder=\"Enter your name...\" value=\"Enter your name...\" onfocus=\"if (this.value == 'Enter your name...') {this.value = '';}\" onblur=\"if (this.value == '') {this.value = 'Enter your name...';}\" title=\"Enter your name...\" \/>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"subscribeunlock_form_column subscribeunlock_50 subscribeunlock_form_right_column\">\n\t\t\t\t\t\t\t<div>\n\t\t\t\t\t\t\t\t<input required=\"required\" tabindex=\"102\" class=\"subscribeunlock-input subscribeunlock_fullwidth\" type=\"text\" id=\"email_8042\" placeholder=\"Enter your e-mail...\" value=\"Enter your e-mail...\" onfocus=\"if (this.value == 'Enter your e-mail...') {this.value = '';}\" onblur=\"if (this.value == '') {this.value = 'Enter your e-mail...';}\" title=\"Enter your e-mail...\" \/>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\n\t\t\t\t\t<div>\n\t\t\t\t\t\t<a href=\"#\" tabindex=\"103\" class=\"subscribeunlock-submit\" id=\"submit_8042\" onclick='subscribeunlock_submit(\"_8042\", \"off\"); return false;'>UNLOCK CONTENT<\/a>\n\t\t\t\t\t\t<img decoding=\"async\" id=\"loading_8042\" class=\"subscribeunlock_loading\" src=\"http:\/\/wizardofbots.com\/network\/wp-content\/plugins\/subscribe-unlock\/images\/loading.gif\" alt=\"\">\n\t\t\t\t\t<\/div>\n\t\t\t\t\t<div id=\"message_8042\" class=\"subscribeunlock_message\"><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"subscribeunlock_confirmation_container\" id=\"subscribeunlock_confirmation_container_8042\"><\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n<p>Then just run the command using node like:<br \/>\nnode crawler.js domain.com<\/p>\n<p>Thats it, it will create a domain.com.txt with all the emails.<\/p>\n<p>Your console will look like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/i.imgur.com\/kZet78O.png\" width=\"1112\" height=\"641\" \/><\/p>\n<p>And emails grabbed like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/i.imgur.com\/gzD4V2b.png\" width=\"723\" height=\"516\" \/><br \/>\nObviously change domain.com by any domain you want to crawl and get emails.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/media3.giphy.com\/media\/aQCCNezRpb9Hq\/giphy.gif\" width=\"1200\" height=\"675\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Whats up everybody! I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies. Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[111,6],"tags":[123,122,39,124],"class_list":["post-284","post","type-post","status-publish","format-standard","hentry","category-nodejs","category-tutorials","tag-domain-crawler","tag-email-extractor","tag-nodejs","tag-roboto"],"_links":{"self":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=284"}],"version-history":[{"count":4,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284\/revisions"}],"predecessor-version":[{"id":633,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284\/revisions\/633"}],"wp:attachment":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=284"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=284"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=284"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}