Experiments in wgetting
Feb. 2nd, 2019 10:04 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
brin_bellwayCurrent experiment: glowfic.com. Status: failure.
A partial test run suggested that I should --reject "*unread=true*","*view=flat*" to save storage space (which *is* very much a concern, given that I like to keep a full copy of my stuff on my smartphone), but unfortunately it still has to *check* each unread=true, of which there are vast quantities. After four days, wget had consumed 30% of my (8 GB) RAM with no end in sight, and my computer was beginning to reach an annoying level of lag when performing its normal activities.
If I had a suitable computer that I could afford to completely dedicate to the task for a while, I might be able to manage it. Unfortunately, all of the spare computers lying around have even less RAM and might just straight-up run out if they tried to do this.
Well, that site was always a stretch goal anyway: I've heard you can already get a fair bit of it in epub format. Maybe someday after a RAM upgrade I'll try again.
A partial test run suggested that I should --reject "*unread=true*","*view=flat*" to save storage space (which *is* very much a concern, given that I like to keep a full copy of my stuff on my smartphone), but unfortunately it still has to *check* each unread=true, of which there are vast quantities. After four days, wget had consumed 30% of my (8 GB) RAM with no end in sight, and my computer was beginning to reach an annoying level of lag when performing its normal activities.
If I had a suitable computer that I could afford to completely dedicate to the task for a while, I might be able to manage it. Unfortunately, all of the spare computers lying around have even less RAM and might just straight-up run out if they tried to do this.
Well, that site was always a stretch goal anyway: I've heard you can already get a fair bit of it in epub format. Maybe someday after a RAM upgrade I'll try again.
