Welcome to Inkbunny...
Allowed ratings
To view member-only content, create an account. ( Hide )
Whippy

ArtHawk link is up again. I think I fixed those FA bugs

by
DOWNLOAD HERE

I moved the FA link parsing code into "arthawk.dll" this time, so it can check the header it gets back when testing urls to see if they actually work. If they fail, you'll get a message box showing you what the link is, and the arthawk.dll process will exit gracefully instead of throwing up a nasty empty variable error. This won't affect the main .exe though.  I also made the default extension "png" for when FA's troggle dork fails to give a real extention to the file. ^.^

I've just tested ArtHawk with these artist's galleries on FA:
Jazzycat
Vincent
Aemi
Rarakie
Zen
Galgard

It worked perfectly and got everything. It also worked fine with InkBunny and DeviantArt.
Heh, as before, feel free to report any bugs or give suggestions! If you care to donate, I accept those as well: https://inkbunny.net/donate.php?user_id=75

If you stumble on any files that won't download or give an error, please try to get it's name or URL, or which artist's gallery you were downloading when it happened. I can't really do much here without testing the problem links myself and seeing what they look like.

My next planned update (after a looooong break and making lots of nice new art) will be adding a CSV file for each artist's files. This will greatly speed up the checking for updates process and should enable you to change the file names on the fly without having to delete anything or start over. I'll also look into auto-adding your watch list, or any user's watch list from all 3 sites.

^^ Happy Downloading folks.

Quick Instructions for getting started:
1.Fill in your user name and pass for IB and FA, DA doesn't need one.
2. Press the "edit artist's list" button to edit each site's list.
3.Copy your watch list from IB (there's an option to show names as text only) or FA. Be sure to clean any extra spaces or blank lines from each list, otherwise that particular name will fail to download. Hit "save" when done.
4.Set your save pattern with those two buttons on the right.
5. Under those two buttons, you can press "set your save path" to select or create a folder where all the files and folders will be created for each site and artist. Otherwise ArtHawk will create folders in it's own directory and download to those.
6.Press "check site" to check the site currently selected in the dropdown menu on the top left, or "check all sites" to do just that. It will login and begin downloading the submissions for each artist.

You can also use renaming by clicking "use renaming", then clicking one of those variable names such as *artist* or *title*. It will be copied to your clipboard, then you can paste it into the "file naming pattern" box below. Those variables will be filled in with that info for each file and the file will be named accordingly. Also, I had to add that string of numbers you usually see in IB's or FA's file name because in some cases that's the only thing that makes the file have a unique name. You can put a character or a couple of characters in the box to the right that will be used as a seperator between the file's title and the string of numbers. Like this for example: Using *title* and "____" as a seperator would give you: "Really Cool Pic____02524551.jpg"
Viewed: 75 times
Added: 6 years, 6 months ago
 
Whitehair
6 years, 6 months ago
Thank you for all your hard work.  I have downloaded this and I will give it a try as soon as I have a bit of time.  <wags>
Whippy
6 years, 6 months ago
Excellent ^_^ I hope it works well for you! You're welcome btw. hehe!
Whitehair
6 years, 6 months ago
So far I have not hit any bugs, but I have two minor suggestions for future features for you:
  1) a pause/resume button
  2) a stop button (rather than having to exit the program to stop downloading)

I am loving this program.
Whippy
6 years, 6 months ago
*nods* Yes, I have been wanting to add that in. It will be in the next version :)
Whitehair
6 years, 6 months ago
MINOR BUG REPORT:

I am using the rename option with the following settings:  *year*_*month*_*day*_*artist*__*title*

An example result from InkBunny is:  2010_00_91_Aaron__Up Late.23870.jpg

It happens on FA too.

Whippy
6 years, 6 months ago
Oh damn, hmmm. Argh, I'll look into it tomorrow. It should be a simple one to fix. Thanks for letting me know.
Whitehair
6 years, 6 months ago
Not a problem.  I would guess it is something very minor that accidentally got tweaked when you fixed a different problem.
Whippy
6 years, 6 months ago
Fixed the date issue, it was a simple mistake on my part heh.
http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
Whitehair
6 years, 6 months ago
Working beautifully!
Whippy
6 years, 6 months ago
^_^ Love to hear it!
JeffyCottonbun
6 years, 6 months ago
Thank you! ^^
Whippy
6 years, 6 months ago
*hugs a bunny*
You're very welcome my friend ^.^
Whippy
6 years, 6 months ago
for you Jeffy ^_^
updatenessesessss: http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
JeffyCottonbun
6 years, 6 months ago
^_^ yippeee!  Thankies!
smaxattax
6 years, 6 months ago
Furaffinity user "Zaush" at around submission 105/153:

" Error: This dynamic variable is blank. If this variable was not intended to be dynamic, remove the % symbols from it.

Specifically: %s2%

        Line#
--->  937: month := %s2%

The current thread will exit.
Whippy
6 years, 6 months ago
Ah ha....god damn I hate FA....
The link looks like this:
https://d.facdn.net/art/zaush/zaush_1122110383_citra-sh...
I'll make a check for that now...
smaxattax
6 years, 6 months ago
Ah, you're right. It downloaded several thumbnails instead of full submissions.

So you aren't just parsing for the "Download" URL in each submission page?
Whippy
6 years, 6 months ago
" So you aren't just parsing for the "Download" URL in each submission page?

Nope, that would be way too much bandwidth usage. I just get the gallery pages. Each thumbnail link is usually just the full file link with .thumbnail added in. Or if it's an swf, it always ends in .swf.gif. The problem is, not all file names have numbers and not all file names have the artist name seperated by an underscore. Zaush's gallery has a shit ton of these around the last half. It's a complete train wreck of file naming -__-
I can't really figure this out atm because i've been working on it too long today and about to throw my keyboard out the window, so I'm taking the rest of the day off. I'll see what I can do tomorrow.
smaxattax
6 years, 6 months ago
Ah, I made a greasemonkey script to do something similar (always show full view by removing .half).

FurAffinity tends to throw up 503s if you request pages too quickly (to prevent excess server load, which is fair), but you'll be fine if you use a single thread and sleep between requests for a fair amount of time.

The way I see it, you can do it slow and steady the manual way (sleeping for 1000ms or so and going single-threaded) for reliability, or try to work out a hybrid of both methods. See this thread: http://www.autohotkey.com/forum/topic63019.html

From that, It appears you can do some sort of exception handling in AutoHotKey. So when you're manipulating thumbnail URLs, if something goes wrong or an exception is thrown up, you can escape it by aborting the current submission and storing its submission ID into an array. Later, the troublesome submissions can be done the slow way.

Anyways, take that well-needed break.
Whippy
6 years, 6 months ago
Couldn't sleep just yet and FA really got my goat :D
http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
Fixed the date issue, and Zaush's art fully downloads now.

Really liking that array idea ^^
Yesh...sleep...break..paint...rawr..*drools*
smaxattax
6 years, 6 months ago
In the middle of a big batch right now and noticed a minor issue.

https://www.furaffinity.net/gallery/0-kado-0/
https://www.furaffinity.net/view/4598002/

This image is .png, but ArtHawk is saving it as .jpg.

As a last resort, you can detect the magic numbers: https://en.wikipedia.org/wiki/Magic_number_(programming...
Whippy
6 years, 6 months ago
I tried DLing his gallery, and so did my friend. We both were able to download his whole gallery on FA. The image you linked here also saves as png. Are you sure you have the right version of the app and it's .dll file? What OS are you using?
smaxattax
6 years, 6 months ago
I'm terribly sorry, user error this time (in more than one way). The file is misnamed on the server (JPEG with .png extension) and I remembered it backwards.

A word of caution when you implement watchlist adding though. I copy-pasted mine and found that FurAffinity doesn't include underscores in a user's URL, even when their username has them (e.g. argon_vile = /argonvile/). You may want to remove underscores automatically from the FA artist list or on-the-fly before downloading.
Whippy
6 years, 6 months ago
Yeah, hehe, Jeffy pointed that out to me pretty quickly today. Man...I hate FA more every day :P
I'll do a fix for that later today. Heh, till then you can do a search and replace on your user list before pasting it into arthawk ^^

Thanks for letting me know! :)
smaxattax
6 years, 6 months ago
Furafinnity user "Frisky-Lime" ~444/507

" Error: This dynamic variable is blank. If this variable was not intended to be dynamic, remove the % symbols from it.

Specifically: %s2%

        Line#
--->  969: month := %s2%

The current thread will exit.


I also had a random %s2% earlier, but I wasn't paying attention and couldn't narrow down what user gallery caused it. You might want to consider outputting an error log.
smaxattax
6 years, 6 months ago
"Gilbhart" also spawns a few of these. FA is looking to be the bane of your existence.
smaxattax
6 years, 6 months ago
I noticed that FurAffinity login is always said to be successful, even if I input junk characters as the username/pass.

Today after doing some random grabs and editing my artist list, I noticed near the end that it was only grabbing clean art from FurAffinity, which resolved itself once I restarted. I don't know why my login failed (could have just been a server fart), but since ArtHawk is unable to detect login failures, it just reverted into guest viewing.
Whippy
6 years, 6 months ago
I noticed that yesterday after BJ told me it only got clean art from FA. It was checking for there being no cookies instead of checking for the presence of "a=" in the cookies. I've fixed that pretty easily. Guess that's what I get for not sleeping enough ^^;

Also, I've made ArtHawk download the actual pages for the submissions now (and parse for the "small_url"), but FA is incredibly slow with downloading each page -_- So I'm thinking of having it fetch the thumbnail url and get the header first before downloading the submission's page itself. I've also made it write a simple .cache log of filles downloaded when they actually complete. It checks this log to see if any of the URLs are present and if not, adds them to the list to be downloaded.
smaxattax
6 years, 6 months ago
Cool. I'll quit bugging 'ya until the new build comes around.

I had thought you might have had to resort to to grabbing individual submission pages because of thumbnail inconsistencies, but it's not a total loss. You can grab extra metadata (like keywords) to stuff into the CSV from the pages. Cool things could be done with it. For instance, I could write a script to inject the images and metadata into a local imageboard (e.g. shimmie) to have your very own custom furbooru.
MichaelJBear
6 years, 6 months ago
Héhé I'm gonna give a try and let you know...
Whippy
6 years, 6 months ago
Try this one, fixed some mentioned issues above: http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
Lemme know if FA gives you anymore trouble...lol..I'm sure it will somewhere. *shakes his fist at troggle dork*
btk6
6 years, 6 months ago
thank you 4 the update.
 
i like hoarding art. this helps a lot.*big draggy hug*
Whippy
6 years, 6 months ago
here's another update lol: http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
I like hoarding art too, isn't it pawsome? ^.^
btk6
6 years, 6 months ago
yes, i have 55 GB Free. i know i cant fill it with jut art. got to have room for game any way.
kenji321
6 years, 6 months ago
again thank you for your hard work
Whippy
6 years, 6 months ago
you're welcome!
updated just now btw: http://dl.dropbox.com/u/43927622/apps/ArtHawk/ArtHawk.zip
fixed some issues mentioned above.
kenji321
6 years, 6 months ago
Yeah but I knew the link was on the way
Whippy
6 years, 6 months ago
^_^ Hehe! I guess I have been kinda consistent like that.
CosmicMutt
6 years, 6 months ago
its not downloadable... :c
Whippy
6 years, 6 months ago
CosmicMutt
6 years, 6 months ago
thank you kind sir!
Knoton
6 years, 6 months ago
Any chance you could modify it to download the user's favorites?
FoxxyFluff
6 years, 6 months ago
Working good for me so fars. Got a couple pop ups but this times didn't lock it up
FoxxyFluff
6 years, 6 months ago
edit founds a problem. When downloading Atsumori999 gallery on FA it doensent download and the main app says found number of thumbnail URLS 290
FoxxyFluff
6 years, 6 months ago
Neva mind a restart fixed it
New Comment:
Move reply box to top
Log in or create an account to comment.