|
General software and network General OS-independent software and network questions, X11, MTA, routing, etc. |
|
Thread Tools | Display Modes |
|
|
|||
your favorite mirroring tool
Hi everyone !
What is your favorite mirroring tool ? wget ? httrack ? rsync ? w3mir ? and why ? wget is cool but if mirroring is interrupted , -k becomes useless and bye to offline browsing .. here httrack excels .. as excels in specifying wildcards .. Please share your thoughts :-) |
|
|||
I mainly mirror files within my LAN as an inexpensive form of backup.
Since my main system runs OpenBSD I just start the ftpd on the server using the defaults. For the clients I use several tools: On Linux systems I usually install lftp which has a mirroring mode. When I need to sort through a large number of files/directories Midnight commander (misc/mc) has an ftp mode that allows tagging of files/directories and selection of the latest/largest files along with recursively moving an entire directory. The 2 panel view is a plus. You can run commands on files in mc. I have a "rainy weekend" thought that I may be able to show a package respository in the panel and pass the tagged files to $ xterm -e sudo pkg_install -i as a quasi graphical package manager in OpenBSD.
Last edited by shep; 4th June 2013 at 10:00 PM. |
|
|||
rsync does what I want.
Regards. |
|
|||
Which begs the following question. daemonfowl, what is it that you are attempting to accomplish? Are you really wanting to mirror files, or back-up files?
|
|
|||
Hi all and thank you very much for sharing opinions and ideas !
Quote:
I'd like to know BSD folk predilection .. but also to know about other mirroring tools .. I use httrack for dynamic websites and wget for static ..some websites refuse to mirror either way (like the undeadly.org ) .. Why mirror ?? please do accept a fool's answer : either to keep it for offline reference .. or for the sake of event chronicling -so to speak .. I'll be happy to know if there is a way to mirror such websites when httrack and wget fail to .. :-) Quote:
|
|
|||
I use rsync, but I haven't looked carefully at how I mirror and backup yet. There are some things I hope to do better someday.
I've generally made do with one way sync. i.e. I take whatever's on one side and replace what's on the other side with that. Eventually I want to merge the two sides taking what's changed from either (or the more appropriate with conflicts resolved interactively). While starting to learn the ML programming language, I noticed in the promotional materials for OCaml that there's a file sync program called Unison written in OCaml that's supposed to handle that well. Someday I'll try it. Another piece I've not looked at yet is keeping old copies of files (akin to VMS, Hammer's builtin snapshotting, or a version control system). With rsync, I'm pretty sure this is simply a matter of supplying the right flag (this has come up on OpenBSD mailing lists) but I haven't tried that yet either. Finally, I have to figure out a better way of syncing a bunch of mail folders, as created by the Gnus newsreader/mail client, on one machine and syncing them in a way that I can run Gnus on the 2nd machine and have all pointers or whatever synced up in a way that it looks the same as on the original machine. I'm guessing this will require understanding how Gnus writes its files and may not be a simple matter of syncing every file involved. But until I can get OpenBSD working on my fast machine (someone added an ACPI fix to a recent current that looks very promising), I'm considering everything sort of interim anyway, so my syncing is kind of sloppy. |
|
|||
So , why do some websites refuse "wget -r -m -k --no-parent" ?
|
|
|||
daemonfowl, you're hijacking your own thread again.
|
|
|||
sorry I'll start a new thread .. despite the fact that the reason behind this very thread is :
that I cannot rely on httrack nor on wget to mirror a few websited of interest (undeadly.org) .. is it a CyberTaboo ? :-) |
|
|||
Quote:
Before posting, you should ask yourself what is the fundamental issue that you would like others to comment upon, & articulate the problem & motivation upfront. |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
What is your favorite font? | guitarscn | Off-Topic | 4 | 1st January 2011 01:46 AM |
your 3 favorite games | welkin | FreeBSD Ports and Packages | 17 | 5th June 2010 08:26 PM |
What's your favorite Window Manager? | welkin | FreeBSD Ports and Packages | 1 | 13th February 2010 10:39 AM |
Favorite x86 CPU manufacturer for... | JMJ_coder | Off-Topic | 12 | 12th July 2008 05:59 PM |