{"id":284,"date":"2017-10-24T03:20:08","date_gmt":"2017-10-24T03:20:08","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=284"},"modified":"2017-10-25T01:33:40","modified_gmt":"2017-10-25T01:33:40","slug":"node-js-full-domain-crawler-email-extractor-source","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","title":{"rendered":"Node.JS full domain crawler + email extractor (source)"},"content":{"rendered":"<p>Whats up everybody!<\/p>\n<p>I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies.<\/p>\n<p>Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step by step:<\/p>\n<p>1.- Create a dir.<br \/>\n2.- Go inside of it.<br \/>\n3.- You need to install roboto and htmlstrip-native with npm.<br \/>\n4.- Create a crawl.js file inside that folder you created.<br \/>\n5.- Paste the source code on it:<\/p>\n\n\t\t<div class=\"subscribeunlock_container\">\n\t\t\t<div name=\"subscribeunlock\" class=\"subscribeunlock_box\" id=\"subscribeunlock_4486\">\n\t\t\t\t<div class=\"subscribeunlock_signup_form\" id=\"subscribeunlock_signup_form_4486\">\n\t\t\t\t\t<div class=\"subscribeunlock_form_row\"><p>Complete your name and email to unlock the code...<\/p><\/div>\n\t\t\t\t\t<div class=\"subscribeunlock_form_row\">\n\t\t\t\t\t\t\n\t\t\t\t\t\t<div class=\"subscribeunlock_form_column subscribeunlock_50\">\n\t\t\t\t\t\t\t<div>\n\t\t\t\t\t\t\t\t<input required=\"required\" tabindex=\"101\" class=\"subscribeunlock-input subscribeunlock_fullwidth\" type=\"text\" id=\"name_4486\" placeholder=\"Enter your name...\" value=\"Enter your name...\" onfocus=\"if (this.value == 'Enter your name...') {this.value = '';}\" onblur=\"if (this.value == '') {this.value = 'Enter your name...';}\" title=\"Enter your name...\" \/>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<div class=\"subscribeunlock_form_column subscribeunlock_50 subscribeunlock_form_right_column\">\n\t\t\t\t\t\t\t<div>\n\t\t\t\t\t\t\t\t<input required=\"required\" tabindex=\"102\" class=\"subscribeunlock-input subscribeunlock_fullwidth\" type=\"text\" id=\"email_4486\" placeholder=\"Enter your e-mail...\" value=\"Enter your e-mail...\" onfocus=\"if (this.value == 'Enter your e-mail...') {this.value = '';}\" onblur=\"if (this.value == '') {this.value = 'Enter your e-mail...';}\" title=\"Enter your e-mail...\" \/>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t\n\t\t\t\t\t<div>\n\t\t\t\t\t\t<a href=\"#\" tabindex=\"103\" class=\"subscribeunlock-submit\" id=\"submit_4486\" onclick='subscribeunlock_submit(\"_4486\", \"off\"); return false;'>UNLOCK CONTENT<\/a>\n\t\t\t\t\t\t<img decoding=\"async\" id=\"loading_4486\" class=\"subscribeunlock_loading\" src=\"http:\/\/wizardofbots.com\/network\/wp-content\/plugins\/subscribe-unlock\/images\/loading.gif\" alt=\"\">\n\t\t\t\t\t<\/div>\n\t\t\t\t\t<div id=\"message_4486\" class=\"subscribeunlock_message\"><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"subscribeunlock_confirmation_container\" id=\"subscribeunlock_confirmation_container_4486\"><\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n<p>Then just run the command using node like:<br \/>\nnode crawler.js domain.com<\/p>\n<p>Thats it, it will create a domain.com.txt with all the emails.<\/p>\n<p>Your console will look like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/i.imgur.com\/kZet78O.png\" width=\"1112\" height=\"641\" \/><\/p>\n<p>And emails grabbed like this:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/i.imgur.com\/gzD4V2b.png\" width=\"723\" height=\"516\" \/><br \/>\nObviously change domain.com by any domain you want to crawl and get emails.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full\" src=\"https:\/\/media3.giphy.com\/media\/aQCCNezRpb9Hq\/giphy.gif\" width=\"1200\" height=\"675\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Whats up everybody! I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies. Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"footnotes":""},"categories":[111,6],"tags":[123,122,39,124],"class_list":["post-284","post","type-post","status-publish","format-standard","hentry","category-nodejs","category-tutorials","tag-domain-crawler","tag-email-extractor","tag-nodejs","tag-roboto"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Node.JS full domain crawler + email extractor (source) - Wizard Of Bots<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Node.JS full domain crawler + email extractor (source) - Wizard Of Bots\" \/>\n<meta property=\"og:description\" content=\"Whats up everybody! I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies. Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/\" \/>\n<meta property=\"og:site_name\" content=\"Wizard Of Bots\" \/>\n<meta property=\"article:published_time\" content=\"2017-10-24T03:20:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-10-25T01:33:40+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i.imgur.com\/kZet78O.png\" \/>\n<meta name=\"author\" content=\"wizardofbots\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"wizardofbots\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/\",\"url\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/\",\"name\":\"Node.JS full domain crawler + email extractor (source) - Wizard Of Bots\",\"isPartOf\":{\"@id\":\"https:\/\/wizardofbots.com\/network\/#website\"},\"primaryImageOfPage\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage\"},\"image\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i.imgur.com\/kZet78O.png\",\"datePublished\":\"2017-10-24T03:20:08+00:00\",\"dateModified\":\"2017-10-25T01:33:40+00:00\",\"author\":{\"@id\":\"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\"},\"breadcrumb\":{\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage\",\"url\":\"https:\/\/i.imgur.com\/kZet78O.png\",\"contentUrl\":\"https:\/\/i.imgur.com\/kZet78O.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/wizardofbots.com\/network\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Node.JS full domain crawler + email extractor (source)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/wizardofbots.com\/network\/#website\",\"url\":\"https:\/\/wizardofbots.com\/network\/\",\"name\":\"Wizard Of Bots\",\"description\":\"Botting and AI community\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/wizardofbots.com\/network\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f\",\"name\":\"wizardofbots\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/\",\"url\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"contentUrl\":\"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g\",\"caption\":\"wizardofbots\"},\"url\":\"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Node.JS full domain crawler + email extractor (source) - Wizard Of Bots","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","og_locale":"en_US","og_type":"article","og_title":"Node.JS full domain crawler + email extractor (source) - Wizard Of Bots","og_description":"Whats up everybody! I was a away for a long time, deep projects and shit, but I am back and will continue doing some course and more freebies. Anyway, here is a brief domain crawler + email extractor I did with Node.JS using roboto library which is cool and easy. So here I go, step [&hellip;]","og_url":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","og_site_name":"Wizard Of Bots","article_published_time":"2017-10-24T03:20:08+00:00","article_modified_time":"2017-10-25T01:33:40+00:00","og_image":[{"url":"https:\/\/i.imgur.com\/kZet78O.png"}],"author":"wizardofbots","twitter_card":"summary_large_image","twitter_misc":{"Written by":"wizardofbots","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","url":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/","name":"Node.JS full domain crawler + email extractor (source) - Wizard Of Bots","isPartOf":{"@id":"https:\/\/wizardofbots.com\/network\/#website"},"primaryImageOfPage":{"@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage"},"image":{"@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage"},"thumbnailUrl":"https:\/\/i.imgur.com\/kZet78O.png","datePublished":"2017-10-24T03:20:08+00:00","dateModified":"2017-10-25T01:33:40+00:00","author":{"@id":"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f"},"breadcrumb":{"@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#primaryimage","url":"https:\/\/i.imgur.com\/kZet78O.png","contentUrl":"https:\/\/i.imgur.com\/kZet78O.png"},{"@type":"BreadcrumbList","@id":"http:\/\/wizardofbots.com\/network\/node-js-full-domain-crawler-email-extractor-source\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/wizardofbots.com\/network\/"},{"@type":"ListItem","position":2,"name":"Node.JS full domain crawler + email extractor (source)"}]},{"@type":"WebSite","@id":"https:\/\/wizardofbots.com\/network\/#website","url":"https:\/\/wizardofbots.com\/network\/","name":"Wizard Of Bots","description":"Botting and AI community","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/wizardofbots.com\/network\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/31f9e486da1c11791d94a861854a2a9f","name":"wizardofbots","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/wizardofbots.com\/network\/#\/schema\/person\/image\/","url":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","contentUrl":"http:\/\/2.gravatar.com\/avatar\/584eebc303f64610559ab9f305f6928d?s=96&d=mm&r=g","caption":"wizardofbots"},"url":"http:\/\/wizardofbots.com\/network\/author\/wizardofbots\/"}]}},"_links":{"self":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284"}],"collection":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=284"}],"version-history":[{"count":4,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284\/revisions"}],"predecessor-version":[{"id":633,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/284\/revisions\/633"}],"wp:attachment":[{"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=284"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=284"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=284"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}