{"id":233,"date":"2016-10-11T22:00:14","date_gmt":"2016-10-11T22:00:14","guid":{"rendered":"http:\/\/wizardofbots.com\/network\/?p=233"},"modified":"2016-10-11T22:31:47","modified_gmt":"2016-10-11T22:31:47","slug":"how-to-mass-download-images-from-multiple-pages-of-tumblr-profile","status":"publish","type":"post","link":"http:\/\/wizardofbots.com\/network\/how-to-mass-download-images-from-multiple-pages-of-tumblr-profile\/","title":{"rendered":"How to mass download images from multiple pages of Tumblr profile"},"content":{"rendered":"

\"\"Hey whatsup fellas. After opening the courses and scripts sections im updating this blog with something new and different. How many times you wanted to make a private gallery from many different tumblr profiles to watch offline? Maybe zero, but with this knowledge you will be able to do it with just 1 line of code!<\/p>\n

So what it is going to do, is using curl to mass scrape and paginate a tumblr profile, so you will get a list of URLs that will be processed with a while loop inside the curl, and then saving it on the folder you are running this command. But first…<\/p>\n

You might need to install cURL in your server, dont worry is easy:<\/p>\n

\n
sudo apt-get install curl<\/code><\/pre>\n<\/blockquote>\n
And if this doesnt works, try doing an apt-get update and then:<\/p>\n
\n
sudo apt-get install libcurl3 php5-curl<\/code><\/pre>\n<\/blockquote>\n
Then just like that,\u00a0type: curl -h \u00a0to see the help, if it display now you have it installed.<\/p>\n
This is the main command you have to type inside the folder where you want to put all the downloaded images:<\/p>\n
curl http:\/\/concept-art.tumblr.com\/page\/[1-7] | grep -o ‘src=”[^”]*.[png-jpg]”‘ | cut -d\\” -f2 | while read l; do curl “$l” -o “${l##*\/}”; done<\/span><\/p>\n
Lets explain the code a little bit: the curl command is requesting the URL\u00a0http:\/\/concept-art.tumblr.com\/page\/1 starting with page 1, then when we put in [ ] and separate a bigger number at the right with a dash – , we will be able to request multiple pages doing a pagination. Then when you add |<\/strong> means that it is separating the commands to run after, so\u00a0grep<\/strong> is going to do a search for\u00a0src attributes<\/strong> and only for those with termination on\u00a0[png-jpg]\u00a0<\/strong>or you can add more doing\u00a0-gif<\/strong> inside the [ ]. Then the\u00a0cut\u00a0<\/strong> command will collect the names, but what it is going to make download eveything will be the while loop, doing a curl on the found urls for the png or jpg images.<\/p>\n
So make sure to add the while loop to the end for downloading, if you only want to see the images grabbed using the following command:<\/p>\n
curl http:\/\/concept-art.tumblr.com\/page\/[1-7] | grep -o ‘src=”[^”]*.[png-jpg]”‘ | cut -d\\” -f2<\/span><\/p>\n
Here is a video that I recorded in order to show you how to do it:<\/p>\n