{"id":233,"date":"2016-10-11T22:00:14","date_gmt":"2016-10-11T22:00:14","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=233"},"modified":"2016-10-11T22:31:47","modified_gmt":"2016-10-11T22:31:47","slug":"how-to-mass-download-images-from-multiple-pages-of-tumblr-profile","status":"publish","type":"post","link":"https:\/\/wizardofbots.com\/network\/how-to-mass-download-images-from-multiple-pages-of-tumblr-profile\/","title":{"rendered":"How to mass download images from multiple pages of Tumblr profile"},"content":{"rendered":"<p><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full\" src=\"https:\/\/media0.giphy.com\/media\/E241gLRHRRiYE\/200w.gif#69\" alt=\"\" width=\"200\" height=\"300\" \/>Hey whatsup fellas. After opening the courses and scripts sections im updating this blog with something new and different. How many times you wanted to make a private gallery from many different tumblr profiles to watch offline? Maybe zero, but with this knowledge you will be able to do it with just 1 line of code!<\/p>\n<p>So what it is going to do, is using curl to mass scrape and paginate a tumblr profile, so you will get a list of URLs that will be processed with a while loop inside the curl, and then saving it on the folder you are running this command. But first&#8230;<\/p>\n<p>You might need to install cURL in your server, dont worry is easy:<\/p>\n<blockquote>\n<pre><code>sudo apt-get install curl<\/code><\/pre>\n<\/blockquote>\n<p>And if this doesnt works, try doing an apt-get update and then:<\/p>\n<blockquote>\n<pre><code>sudo apt-get install libcurl3 php5-curl<\/code><\/pre>\n<\/blockquote>\n<p>Then just like that,\u00a0type: curl -h \u00a0to see the help, if it display now you have it installed.<\/p>\n<p>This is the main command you have to type inside the folder where you want to put all the downloaded images:<\/p>\n<p><span class=\"lang:default decode:true crayon-inline\">curl http:\/\/concept-art.tumblr.com\/page\/[1-7] | grep -o &#8216;src=&#8221;[^&#8221;]*.[png-jpg]&#8221;&#8216; | cut -d\\&#8221; -f2 | while read l; do curl &#8220;$l&#8221; -o &#8220;${l##*\/}&#8221;; done<\/span><\/p>\n<p>Lets explain the code a little bit: the curl command is requesting the URL\u00a0http:\/\/concept-art.tumblr.com\/page\/1 starting with page 1, then when we put in [ ] and separate a bigger number at the right with a dash &#8211; , we will be able to request multiple pages doing a pagination. Then when you add <strong>|<\/strong> means that it is separating the commands to run after, so\u00a0<strong>grep<\/strong> is going to do a search for\u00a0<strong>src attributes<\/strong> and only for those with termination on\u00a0<strong>[png-jpg]\u00a0<\/strong>or you can add more doing\u00a0<strong>-gif<\/strong> inside the [ ]. Then the\u00a0<strong>cut\u00a0<\/strong> command will collect the names, but what it is going to make download eveything will be the while loop, doing a curl on the found urls for the png or jpg images.<\/p>\n<p>So make sure to add the while loop to the end for downloading, if you only want to see the images grabbed using the following command:<\/p>\n<p><span class=\"lang:default decode:true crayon-inline \">curl http:\/\/concept-art.tumblr.com\/page\/[1-7] | grep -o &#8216;src=&#8221;[^&#8221;]*.[png-jpg]&#8221;&#8216; | cut -d\\&#8221; -f2<\/span><\/p>\n<p>Here is a video that I recorded in order to show you how to do it:<\/p>\n<p><iframe loading=\"lazy\" width=\"720\" height=\"540\" src=\"https:\/\/www.youtube.com\/embed\/FA2p0ACmP8M?feature=oembed\" frameborder=\"0\" allowfullscreen><\/iframe><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Hey whatsup fellas. After opening the courses and scripts sections im updating this blog with something new and different. How many times you wanted to make a private gallery from many different tumblr profiles to watch offline? Maybe zero, but with this knowledge you will be able to do it with just 1 line of [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[107,108,110,109],"class_list":["post-233","post","type-post","status-publish","format-standard","hentry","category-tutorials","tag-curl","tag-download","tag-mass-download","tag-tumblr"],"_links":{"self":[{"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/233","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/comments?post=233"}],"version-history":[{"count":10,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/233\/revisions"}],"predecessor-version":[{"id":243,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/posts\/233\/revisions\/243"}],"wp:attachment":[{"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/media?parent=233"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/categories?post=233"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wizardofbots.com\/network\/wp-json\/wp\/v2\/tags?post=233"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}