1. Store
  2. Apps
  3. Hardware
  4. Support
  5. Solutions

ClearFoundation

Forums
Welcome, Guest
Remote FTP access through NAT router - SOLVED
(1 viewing) 1 Guest
Go to bottomPage: 1
TOPIC: Remote FTP access through NAT router - SOLVED
#5837
Remote FTP access through NAT router - SOLVED 4 Years, 8 Months ago  
This is my first post on the ClearOS forum. I'm the volunteer webmaster of our local amateur theatre. We think ClearOS is a great product, but have had one very bad initial experience which nearly put us off using it at all. We have finally worked out a way round what looked at one point like a show stopper. It seems from other forum posts that others have had similar difficulties in getting inbound external connections to work via FTP, when the ClearOS server is behind a NAT router and firewall.


** The problem

I and another volunteer colleague have been going nuts for ten days trying to set up a new ClearOS 5.1 server at my home, which would act as a test site for a replacement server for our local amateur theatre. It needs to allow remote FTP access, via Dreamweaver, Windows Explorer or other FTP clients to:

  • the main public theatre website;

  • a separate (password protected) website, viewable only by members, running Joomla and extensions;

  • at least one separate flexshare;

  • and home directories for a small number of users who want remote access to their files on the theatre server from their home computer, preferably using a Windows Network Place on their home machine.

We wanted to prototype this using a test server machine at my home, behind a NAT and firewall home router on a dynamic public IP address, referenced by a dyndns.org dynamic DNS domain name.

In the course of doing so, I found that many other forum users have a similar question about getting FTP to work in this way, and with help from my colleague we tried all the suggested solutions. None worked fully, or adequately for what we need, and most made no difference to our particular problem.


** Initial partial solution - open TWO ranges of passive FTP ports on the router

The home folder access worked remotely on port 21 'out of the box'.

But flexshare and website FTP access via port 2121 would just NOT work, except in FileZilla, and even that didn't work to begin with.

Connecting via FileZilla from a remote client didn't work until we had discovered the obscurely documented requirement to open TWO sets of ports on the router 'return paths' for passive FTP: by default in the ClearOS ProFTP setup, these are 60000-61000 (for port 21 FTP connections to home folders), and 65000-65100 (for port 2121 connections to flexshares).

It was only after working out that there are TWO proftpd.conf files and reading them that we realised there are these TWO passive port ranges that need to be opened, and we had initially only found one set of port numbers identified in forum posts and opened only those. (The files are /etc/proftpd.conf, and /etc/proftpd.d/flex-2121.conf ).

Other FTP clients - including Dreamweaver and Windows Explorer on both XP and Vista - would connect, and change to the right folder on the server, but the connection would then hang and time out, rendering them unusable. But we couldn’t see why.


** Symptoms which led to our solution

Dreamweaver could connect from a machine on the local network using the ClearOS machine name or local IP address, but NOT using the external xxxx.dyndns.org address.

After MANY wasted hours, including four clean reinstalls of ClearOS to fix errors we couldn't recover from after trying all sorts of changes to file ownerships and permissions in the flexshare and website folders, we worked out a solution, and (we think) the reason why it works.

It seems that in its default configuration in ClearOS, ProFTP in passive FTP mode responds with the LOCAL IP address of the ClearOS server. We noticed this in the FileZilla FTP log, where at one point the dialogue goes:

Command: PASV
Response: 227 Entering Passive Mode (192,168,2,11,254,3).
Status: Server sent passive reply with unroutable address. Using server address instead

We noticed that the 192,168,2,11 in the above response matched the local IP address 192.168.2.11 of the ClearOS server. (I don't know what the numbers 254, 3 signify, and they are different for different connections.)

FileZilla seems to be clever enough to substitute the server external (router) public IP address (which it already knows from the initial connection) instead of the local address in ProFTP's response, and continues to the conclusion of a usable connection.

Windows Explorer and Dreamweaver don't do the same, so from the point where this response comes back, they can no longer contact the ProFTP server. From an external network they need the router public IP and can't connect to a 'local' address, which is unreachable because it is not on THEIR local network. At this point, where Dreamweaver has already connected and changed to the correct folder on the server, it waits for a long time saying ‘Retrieving remote folder information about <foldername>’ and then times out.


** Our solution

After many false starts and dead ends over ten days and many hours of reading forum posts, ProFTP and ClearOS documentation, we've found the following simple (but maybe not permanent) solution - putting a MasqueradeAddress directive in the ProFTP conf files.

This directive tells ProFTP to return the specified url or IP number instead of the local machine ID in its responses.

So I put the line

MasqueradeAddress xxxx.dyndns.org

in both the /etc/proftpd.conf and /etc/proftpd.d/flex-2121.conf files near the beginning, shortly after the Servername directive (substitute the actual yourdomain.example.com domain you use instead of xxxx.dyndns.org).

If you have a static external public IP address, then you can put that in the directive, instead of a url.

Just putting it in the proftpd.conf file did NOT work, and it may not even be needed there. But when I'd added it to the flex-2121.conf file it then DID work for remote access to website and flexshares.

I could see the difference when I watched the FileZilla FTP client command pane as a remote connection was established to a flexshare. Instead of

227 Entering Passive Mode (192,168,2,11,254,3)

the first four numbers changed to the public IP address of the router.

This change may not survive a reboot of the ClearOS server, or even a change via webconfig in the FTP settings followed by a restart of the FTP service, but for us it has provided, for the first time, a usable workaround.

Further experiment is needed to check whether a reboot or reconfiguration removes the MasqueradeAddress line from the .conf file.


** Use the right form of address to reach the FTP flexshare folders on the server

To access the flexshares and website root folder, one needs ALSO to use the right form of address in the FTP client. The documentation seems to suggest that either of:

ftp://username@flexsharename.yourdomain.example.com:2121 OR
ftp://username@yourdomain.example.com:2121/flexsharename

should work in a browser or Windows Explorer (and you will be prompted for a password, and sometimes for the username again).

For the website root folder, the flexsharename will be in the format yourdomain.example.com

We have never been able to get the first format to work (though perhaps it might if the server itself has a direct public IP address on the Internet?). We don't see how it could be routed through a NAT firewall.

In the second format, most of the time the connection will fail if you omit the /flexsharename after the port number 2121 since the server folder /var/flexshare/shares is not readable by the ftp server daemon (but may be if the connection is made with a username that has read access).

In Dreamweaver, if you omit the 'Remote folder' field which tells Dreamweaver which folder on the server to access on connection, the connection will proabably fail for the same reason - and always does for us.

Oddly (and I don't understand how it works), FileZilla WILL happily connect to the flexshare/shares folder and show all the flexshares, main and virtualhost web root folders in the Remote pane. However, it can only open the main website folder yourdomain.example.com folder. Must be a matter of user:group ownership of folders and/or permissions.

One can also enter the password into the browser address bar, but this leaves the password 'in clear' in the Most Recently Used drop down list which seems insecure. The format is:

ftp://username:password@yourdomain.example.com:2121/flexsharename

** FTP and Flexshare Server Names

In the webconfig Server/FTP Server setup, it isn't very clear what to enter for either the FTP Server Name or the Server/Flexshares FTP tab where it asks for the Server URL. We have tried at different times the machinename, the bare external domain name which points to the router domain.example.com (in our case xxxx.dyndns.org) or the fully qualified external domain name machinename.domain.example.com. We had hoped that the second or third versions would force ProFTP to return the public domain address IP, not the local address, but this seems not to happen.

Whichever we have tried, the same connection failure occurred in Dreamweaver and Explorer before we set the MasqueradeAddress - ProFTP seemed still to be responding with the local network IP address after receiving the PASV command from the FTP client.

** A suggestion to the ClearOS team

Could the documentation be made clearer about the clever but confusing way the FTP server is set up, and could the setup via webconfig include the MasqueradeAddress directive or some even better equivalent that I don't know about?

Either or both would have saved me and many other potential users of this otherwise VERY promising product hours of frustration!

Thanks anyway for what is, in all the other respects we have looked at, appearing as if it should do just what we want.
JohnMcC
Fresh Boarder
Posts: 9
graphgraph
User Offline Click here to see the profile of this user
Last Edit: 2010/02/21 18:15 By johnwmcc.
The administrator has disabled public write access.
 
#5869
Re: Remote FTP access through NAT router - SOLVED 4 Years, 8 Months ago  
Thanks for posting! very in depth

Agreed a masquerade option for standalone servers (who are unable to determine the WAN IP as they are only aware of the gateway) would be a good option. Without some additional scripting / trickery it is not possible for protftpd to know what the WAN IP should be if behind NAT. On the client end, some FTP clients are clever enough to turn it back into a WAN IP if they receive a local IP (such as Filezilla) but that certainly isn't the norm

FTP as you have found is a finicky protocol to get working the way you want! hindered by all sorts of firewall misconfigurations, hopefully your post will help others from suffering the same problems

As an aside for information the PASV command below
227 Entering Passive Mode (a1,a2,a3,a4,p1,p2)
where a1.a2.a3.a4 is the IP address and p1*256+p2 is the port number
Tim Burgess
Moderator
Posts: 6643
graph
User Offline Click here to see the profile of this user
The administrator has disabled public write access.
 
#5878
Re: Remote FTP access through NAT router - SOLVED 4 Years, 8 Months ago  
Thanks for the explanation of the parameters p1 and p2 - I guessed that they somehow referred to the random passive port chosen by the server, but couldn't work out how.

As a PS, we found that it was also possible to set up separate FTP Network Places in Windows XP (or Network Locations in Vista), so that a click on the link in Windows Explorer will open in the flexshare folder, or the website root folder, specified in the :2121/flexsharename portion of the FTP address, or in the user's home folder if using port 21 and not using anything more than ftp://username@yourdomain.example.com in the specified FTP address. The password can be saved.

What IS the best thing to put in the FTP Server Name and Server URL fields in this scenario of a standalone server behind a NAT router?
JohnMcC
Fresh Boarder
Posts: 9
graphgraph
User Offline Click here to see the profile of this user
Last Edit: 2010/02/21 18:27 By johnwmcc.
The administrator has disabled public write access.
 
#7524
Re: Remote FTP access through NAT router - SOLVED 4 Years, 7 Months ago  
Thanks much, John McC.

I found this after I gave up and just put the FTP on the server (in the system I am working on there are 2 CC boxes, one behind the other. It's getting late and it has to work tomorrow, so I gave up the search that would have yielded the "MasqueradeAddress" directive). I might try it later, thanks to your in-depth reporting. Then again, I might not, all the testing required looks fairly lengthy, it may be more than I can afford.

I have a question, if anyone is willing to try to answer- On mine, I can't get to Flexshares via FTP at all. Is this because (when I was trying random things) I set the FTP server to port 2121? As it happens, this is the behavior I want, for the moment- I have a user I wish to give FTP access to, but only to one directory, not to anything else, especially not the allusers share. I share your frustration with the documents, it has been astonishingly difficult to discover whether that is even possible, and if so, exactly how it is done.

One thing I could not make work- SFTP. I broke a lot of stuff trying to limit an SFTP login to one directory. Oh well, at least for now.

Oh, and thanks for the comment about the possibility of proftpd.conf being overwritten, and the pointer to "flex-2121.conf". I didn't open up both ranges of passive ports, just changed proftp.conf to use 65000-65100. I will have to test to see what happens after reboot, and have a look at the other configs...
Phil Nelson
Fresh Boarder
Posts: 2
graphgraph
User Offline Click here to see the profile of this user
The administrator has disabled public write access.
 
#7595
Re: Remote FTP access through NAT router - SOLVED 4 Years, 7 Months ago  
I tested a reboot, the proftpd.conf changes were not overwritten.

Anyone know if there is a reference to what the FTP port ranges are officially supposed to be?
Phil Nelson
Fresh Boarder
Posts: 2
graphgraph
User Offline Click here to see the profile of this user
The administrator has disabled public write access.
 
#7607
Re: Remote FTP access through NAT router - SOLVED 4 Years, 7 Months ago  
As far as I could find out, normal FTP uses port 21, and port 20 for Active return path (always one lower than incoming port). Passive return path can (by default) be anything above 1000-and something (maybe >1024?), or can be restricted to a narrower range by the .conf file..

ClearOS uses ports 21 (& 20 for active FTP return port) for access to home directories, and 2121for access to flexshares, (2120 for active return path for them), and 60000-61000 and 65000-65100 for Passive return paths by default for home directories and flexshares respectively - these return ports are set in proftpd.conf and flex-2121.conf.
JohnMcC
Fresh Boarder
Posts: 9
graphgraph
User Offline Click here to see the profile of this user
The administrator has disabled public write access.
 
Go to topPage: 1
  get the latest posts directly to your desktop