Please help! Rsync/Cygwin/Putty - totally stuck

@tentimes:

What I'm trying to do is Rsync the whole linode server to my windows machine, as a backup. So for this I assumed I would need cygwin with rsync installed.
That should definitely work, yes. There's also a convenient Windows packaging of rsync/cygwin - cwrsync (by the same guy who publishes copssh as an ssh-only package) that you can use if you don't want to bother maintaining a cygwin installation just for this. Another option would be to use unison, which has a nicer GUI on the Windows side, and while it supports two way synchronization, works fine for one-way as well. Either tool will have the same issues (and solutions) when it comes to the underlying ssh channel as below though.

A few caveats in terms of backups though - there's an imperfect match between filesystems in terms of Linux and Windows (in terms of file ownership for example), so while this will work just fine for basic backups of your user files, I'm not sure I'd expect it to work so well for a bare-metal recovery. Also, there are various files on your Linux server (e.g,. /proc) that you'll want to be sure to exclude if you're literally trying to rsync your entire Linode filesystem. So if by "whole Linode … backup" you are expecting to be able to restore back to a brand new Linode, it may not quite do what you want. But if you just expect to be able to restore your own files on top of a distribution after a failure or to roll back, it is probably fine. Depends on your goals. There are other threads here in regards to Linode backups in general, and various options, that you can look for.

For myself, on critical machines I use Linode's own backup service for bare-metal recovery, and then maintain my own backups to ensure access to all of my own non-distribution files on a much finer granularity basis. Whatever your goals, be sure to run tests on the restoration process before you actually need it.

> (1) I can't seem to figure out the SSH command to open the connection. openSSH was installed with cygwin, and ssh username@linodeIP would work normally, but I have SSH on port X and I need to include the keyfile (locally referenced on windows machine) as well. Putty works fine, I can already connect as root with that.
As a general point for looking up help, if you treat your cygwin/openssh/rsync installation as being the same as on Linux, you'll go a long way. That includes looking for man pages, help files, etc.. as any instructions will typically be usable even under Windows, as long as you work in a cygwin bash shell.

For the basic ssh connection, use the "-p" option with the ssh command line client to specify a port, so something like "ssh -p X user@linode". Note that if copying files with scp, the option is "-P" (capital) since -p is used for preserving attributes.

The key is a separate issue. While there might be some way to get putty and openssh to share a key, personally I would suggest just generating a separate key for openssh (if using cwrsync it probably did it already). Under a cygwin shell, run "ssh-keygen" to generate a new key, which will be stored under ~/.ssh (e.g., just like Linux). Then transfer the idrsa.pub file (using, for example, putty) over to your Linode and append it to your user's authorizedkeys file (in ~/.ssh on your Linode). You can have as many keys as you like in there. That will then permit access to your openssh key on your Windows machine in addition to putty's key.

Now, when using rsync, it's going to run ssh for you under the covers, so you need to pass along the same option. In rsync's case you use the "–rsh" option to specify the client command to run and you'll have to quote it since it has spaces - something like "--rsh="ssh -p X"" (excluding the outer quotes). Unison separates the command (-rshcmd) and args (-rshargs) so you'd use something like "-rshargs "-p X"".

However, it's simpler to just put your client requirements in the openssh client configuration file, since then it can get used automatically by any use of ssh. Information like your remote username (if it differs from your Windows account), port to use and any other options can be placed in the ssh configuration file (~/.ssh/config) so that when you use the host "linode" it automatically configures the right settings. This will then work automatically with anything that uses ssh - either you directly, rsync, unison, etc… without having to specify extra options to any of those programs.

Just like under Linux, under cygwin ~ is your home directory. If you're in a cygwin bash shell you can use it directly, but if using Windows tools you can just navigate to the home location. It differs a little depending on how you installed cygwin, but should be able to use "cygpath -w ~" in a bash shell to expand to the Windows path.

> (2) I'm not sure how I would mount a local directory with cygwin so that I could then use rsync to copy stuff form the server to the local (windows machine) local directory.
You don't actually mount anything. rsync works over the ssh connection (which it makes for you under the covers). So when, for example, you say something like "rsync linode:/somepath localpath" it will make an ssh connection to machine "linode", then sync files from remote location "/somepath" to local location "localpath".

– David

David - thanks a million for spending so much time in writing this! Reading through it today and will post back about how I get on.

PuTTYgen allows you to open an existing .ppk Putty-format key and export it to OpenSSH format. Just sayin'.

@rsk:

PuTTYgen allows you to open an existing .ppk Putty-format key and export it to OpenSSH format. Just sayin'.

Very handy! Just logging in now to put David's suggestions to use, so will also try Puttygen, thanks rsk :)

@rsk:

PuTTYgen allows you to open an existing .ppk Putty-format key and export it to OpenSSH format. Just sayin'.
Well I did mention I was sure there was some way to do it. Of course, exporting isn't quite "sharing", so you'd have to re-export if for some reason you need to regenerate the putty key. So I'd probably still recommend just acknowledging that they are different up front and treating them as such.

I admit it's personal preference though…

-- David

PuTTY sucks for a billion reasons, search for win32 alternatives like Tunnelier. UI makes sense, doesn't use the registry for storage and it's a lot faster over long distances due to tcp rountrips, even without server HPN-patch. Support is friendly even to freeloaders like me.

No I don't work for them, just remember my own PuTTY pain :-)

– p

The one and only problem I've had with PuTTY was trying to get openssh keys to work with it. I banged my head against the wall for about an hour before I was able to come up with the proper search terms in google that brought me to a post about converting the openssh keys through PuTTYgen to work with PuTTY.