Download/Backup a whole website
Page 1 of 1
BettyShikle




Posts: 2737
Location: Tardland
PostPosted: Tue, 16th May 2017 20:56    Post subject: Download/Backup a whole website
Hello,

yes,it is one of THOSE topics. I need a tool or a solution in general for doing said task and i can´t figure it out myself somehow. Tried out WinHtTrack but it didn´t work out at all, runs for a bit and crashes and no files written for whatever reason.

I feel like its 2002 again and i´m doing my first steps on the internet. Any ideas?

The Website in question is www.swgsource.com, they fiddle around with the leaked SWG Sourcecode and try to rebuild it while offering tons of guides aswel as premade server-packages in VM format to fool around. I just want to be able to have a mirror of the site on my HD to read trough in case anything happens to them but HOW ? Sad

Thank you
Back to top
Nalo
nothing



Posts: 13508

PostPosted: Tue, 16th May 2017 21:19    Post subject:
⁢⁢


Last edited by Nalo on Wed, 3rd Jul 2024 06:00; edited 2 times in total
Back to top
Bob Barnsen




Posts: 31974
Location: Germoney
PostPosted: Tue, 16th May 2017 21:20    Post subject:
Laughing


Enthoo Evolv ATX TG // Asus Prime x370 // Ryzen 1700 // Gainward GTX 1080 // 16GB DDR4-3200
Back to top
paxsali
Banned



Posts: 18352

PostPosted: Tue, 16th May 2017 21:28    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:12; edited 1 time in total
Back to top
BettyShikle




Posts: 2737
Location: Tardland
PostPosted: Wed, 17th May 2017 01:00    Post subject:
Nalo wrote:
503 - what have you done ?

it was kaputt but is ok now Mr. Green

paxsali wrote:
It's not even possible.

why


paxsali wrote:

Now, I don't know what hardware costs in Poland, I guess it's cheaper because everything is stolen from Germany and resold...
Back to top
Guy_Incognito




Posts: 3433

PostPosted: Wed, 17th May 2017 07:40    Post subject:
Because web sites are not a bunch of HTTP files laying on the server anymore. Now, HTTP files get generated dynamically (using data from i.e. remote database, composing different layout depending on the browser mobile vs desktop) on request by server-side scripts. So this definitely is not 2002. anymore Very Happy
Back to top
The_Zeel




Posts: 14922

PostPosted: Wed, 17th May 2017 09:40    Post subject:
Well it is possible if you have admin access and just download the whole site including databases via FTP or SFTP, but as a simple user/visitor, no.
Back to top
mtj




Posts: 2315
Location: Austria / Finland
PostPosted: Wed, 17th May 2017 10:02    Post subject:
You could create a bot which accesses every link and captures the returned data Smile
Some stuff is more difficult to capture, but I did something like this in one evening (I could not get around dynamically loading pages though. Ie. it loads the data normally, but it then requires scrolling with mouse to generate more of the page) (This was a rare case in my case and I just did this part manually, if I cameup with such page I simply manuially scrolled down on the page and let the crawler continue)

In my case I did not access EVERYTHING, I knew what I was looking for, so you might also need some constraints.

I did this with python as my first ever python... application Razz


Intel Core I-9 9900K @ stock, ASUS Z 390A MoBo, 32GB 3.6GHz, Zotac 3090 | BeQuiet 1000W PSU, Be Quiet Dark Base Pro 900 case, 49" Samsung G9
Back to top
Mister_s




Posts: 19863

PostPosted: Wed, 17th May 2017 10:21    Post subject:
It's illegal to steal a website.
Back to top
PumpAction
[Schmadmin]



Posts: 26759

PostPosted: Wed, 17th May 2017 10:56    Post subject:
It's just a stupid forum generated on server side, why wouldn't your standard website grabber not work? You just need to let it login first and probably run it with admin rights. Also WinHtTrack can be a bitch to setup, maybe follow some simple online guide?

paxsali wrote:
It's not even possible.

Don't listen to this IT illiterate monkey, he knows nothing about the web. Jeez, remember when you told me how you thought that authentication happens on client side? Twisted Evil


=> NFOrce GIF plugin <= - Ryzen 3800X, 16GB DDR4-3200, Sapphire 5700XT Pulse
Back to top
Janz




Posts: 13998

PostPosted: Wed, 17th May 2017 12:56    Post subject:
shouldnt work cause its database assisted and generated stuff

for normal websites i used teleport pro back in the days, but i doubt it will work with that
Back to top
Nalo
nothing



Posts: 13508

PostPosted: Wed, 17th May 2017 13:54    Post subject:
⁢⁢


Last edited by Nalo on Wed, 3rd Jul 2024 06:00; edited 2 times in total
Back to top
Guy_Incognito




Posts: 3433

PostPosted: Wed, 17th May 2017 14:13    Post subject:
Even better: printscreen!
Back to top
PumpAction
[Schmadmin]



Posts: 26759

PostPosted: Wed, 17th May 2017 15:10    Post subject:
Janz wrote:
shouldnt work cause its database assisted and generated stuff

for normal websites i used teleport pro back in the days, but i doubt it will work with that

Wat?!The database generates the site on the backend and delivers to you a ready webpage. Your downloading tool should of course fetch the sites and rewrite the links but as I said, why shouldn't this work? You should end up with a static copy of the page. Nothing will be interactive but the data should be there


=> NFOrce GIF plugin <= - Ryzen 3800X, 16GB DDR4-3200, Sapphire 5700XT Pulse
Back to top
Veki




Posts: 381
Location: Croatia
PostPosted: Wed, 17th May 2017 18:04    Post subject: Re: Download/Backup a whole website
BettyShikle wrote:
Tried out WinHtTrack but it didn´t work out at all, runs for a bit and crashes and no files written for whatever reason.

I tried it also with WinHtTrack and had no problems in copying the site. The only problem is I have no account on the site so couldn't copy forum threads.

Is anything written in "hts-log.txt" (should be in project folder)?

I used this version :
http://download.httrack.com/cserv.php3?File=httrack_x64-noinst.zip


Beware of he who would deny you access to information, for in his heart he dreams himself your master.
Commisssioner Pravin Lal
"U.N. Declaration of Rights"
Back to top
paxsali
Banned



Posts: 18352

PostPosted: Wed, 17th May 2017 20:20    Post subject:
⁢⁢


Last edited by paxsali on Thu, 4th Jul 2024 21:12; edited 1 time in total
Back to top
Janz




Posts: 13998

PostPosted: Wed, 17th May 2017 22:39    Post subject:
and again a very sad day: i have to say pax is right Wink
Back to top
PumpAction
[Schmadmin]



Posts: 26759

PostPosted: Wed, 17th May 2017 22:55    Post subject:
Y'all better be fucking drunk!


=> NFOrce GIF plugin <= - Ryzen 3800X, 16GB DDR4-3200, Sapphire 5700XT Pulse
Back to top
VGAdeadcafe




Posts: 22230
Location: ★ ಠ_ಠ ★
PostPosted: Thu, 18th May 2017 04:02    Post subject:
lmao they must be trolling Laughing
Back to top
Page 1 of 1 All times are GMT + 1 Hour
NFOHump.com Forum Index - Applications
Signature/Avatar nuking: none (can be changed in your profile)  


Display posts from previous:   

Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB 2.0.8 © 2001, 2002 phpBB Group