FTP Upload Issues

Hello everyone. I'm hoping to make contact with someone who has seen this issue before.

I'm trying to set up probably the simplest web site you've ever seen. It is one page, populated with a few charts and graphs and some text. This is my first, and I did want to keep it simple.

I have a weather station that continuously gathers data parameters, temperature, humidity, rain, wind, etc... and wirelessly transmits to a receiver that feeds it to the computer. There, a program automates charts, graphs, data storage, etc... and feeds data to Weather Underground and Citizen's Weather Observer Program (CWOP) on the net. (Links below) It also will generate a simple web page for me to upload to a web site. In full automation, it will generate a new page with new chart data every 15 minutes and upload it via FTP.

Here's where the problem starts. I can't get FTP to work from this location. Symptomatically, using FileZilla or any other FTP client program, I can successfully log in to the server, list directories, change directories, and even upload small HTML files of ~500 bytes or less. However, 1,500 bytes or larger (that's the next size up in the collection of parts for this web page), and it appears as though the client application sends the complete file and waits, only to time out. The remote directory is populated only by the file name of the sent file containing 0 bytes.

The Unix server is using passive mode, simple FTP. The support staff is worthless.

I had someone else in another location log in as me, and they were able to upload successfully.

I am using a router on a LAN, but in troubleshooting the problem, I have bypassed the router and connected directly to the WAN at the DSL modem. Same problem. Same symptoms.

I really don't think it has anything to do with the issue, but in the interest of full disclosure I'm running the latest iMac with OS X Lion Version 10.7.4 and I've tried it on on 3 separate computers and 2 hosting services without success. The friend who was able to upload to the account is in a different city with a different ISP running Windows 7. Every other network job that has involved uploading services, such as SmugMug, etc... has worked without fail. This one just baffles me.

Anyone got any idea on this one? I did call the ISP and got the standard "everything looks OK here" treatment. They offered to sign me up for a very expensive technical support plan.

Weather Underground
http://www.wunderground.com/cgi-bin/findweather/getForecast?query=zmw:72512.2.99999

CWOP
http://www.findu.com/cgi-bin/wxpage.cgi?call=EW0095&last=12

It is really hard to say what the issue might be; it could be anything from antivirus software on your end blocking something, your ip range being blacklisted by the host, etc. Does your host have an administration panel you could use to log in and upload your files? Other than that, I don't have an suggestions. Other than seeing if you can ftp to someone else's server. If you can do that, it would at least eliminate your end as the likely problem.

Thank you for your considerate response. Really good suggestions! I've thought of testing FTP by uploading to a friends computer, but I just haven't done it yet. After reading your suggestion concerning uploading by the host's control panel, I used that method to successfully upload all of the page elements. That gives me the opportunity to futz around with the host's file system and try to get my page to load at http://bullwinkus.com, but it won't help with the automated uploads every 15 minutes that I need to keep the page updated with current weather.

I can't seem to get the URL to default to index.html which assembles the various elements. It still wants to load the goofy spam page, Home.html which I deleted, they stuck on me while I didn't have an index page. I originally set up my page to be in subdirectory /WS/ but moved it to root trying to get it to default to it. Then I reloaded it all again at the root level, because it wasn't working. It still doesn't work. Guess I'll have to open another ticket with the feckless support staff.

Is there a client that logs the ports used in passive mode?

I can see with FileZilla's log that the server is reporting the same IP address always and only changes the port. If I could be certain that my client was following suit and transmitting on the correct port, then, armed with this information, I might be able to persuade my ISP that their firewall is causing my problem.

Thanks again,
Herb

I am a bit confused. Do you mean that you are trying to ftp every 15 minutes when your code is updated? That seems clunky to me, and maybe you have a good reason for doing it this way. If it were me, I would have whatever code was necessary to do the update live on the server. Maybe that is something we can help you with?

Also, I can't imagine that the post would be changing all of the time. That seems weird also.

Is there a client that logs the ports used in passive mode?

On your system, or on your host? I doubt very much that you will be able to install any kind of monitoring software on your host since it will likely violate any TOS.

On my system of course. I meant a client like FileZilla that can log the ports used to send on, as set by the remote server using passive mode. Maybe FileZilla can do it, but I haven't found out how.

The weather data process is live reported to Weather Underground. Well, actually uploaded about every 3 seconds. WU still displays a static page.

Here's a link to another person's pages who use the same software. I think they are using templates with the weather software processing tags before uploading, but it is still a process whereby static pages are replaced frequently through FTP.

I personally think it would be much better to just feed the data to an online program that would continuously update live charts, but I don't know of any.

I think there would be a considerable market for such software.

Avon Weather - Notice the Graphs link in the header. That's where most of the data is.
http://www.avon-weather.com/

This is the static page I loaded earlier today, now out of date, time wise. You can see why it must be refreshed periodically.
http://www.bullwinkus.com/index.html

The weather software I'm using is called WeatherCat from Trixology. It is very sophisticated at collecting data and charting it, and it also handles the web page generation and uploading via FTP. It can also work with templates using tags.

If you go to WUnderground using the link in my first post, look for Wundermap™ under the Maps and Radar Tab. Each one of those circles on the US map represents a private weather station that reports to WU. There are more in England, Europe, etc... They are generally very sophisticated installations, reporting very accurate data and sharing it freely.

Here is a link to a recent screen shot of my desktop running WeatherCat. It runs 24/7. This should give you an idea of the kinds of charts the program is capable of putting online using templates and tags.

I think I am not explaining myself well enough. The weather underground provides an API to programmatically retrieve their data. Instead of a static page, you would need some scripting ability to retrieve the necessary data to update your page on your host. That is how pages are updated in real time. There is no uploading involved.

If I get some time later, I will see if I can throw together a sample of how this would work.

Are you talking about CGI scripting? The WeatherCat software supports that.

WU does have live data on their pages, but not live charts. Their API is rather limited.

Live chart generation is what I assumed you meant in post #4. Sorry.

New development on my FTP upload issue. Had a friend bring over his own Windows 7 computer and he was able to connect and upload to the host server with no difficulty. He connected through the LAN via WiFi. So now, I'm looking for something that might be an issue in the default configuration with my iMacs.

Herb

I would guess it is a firewall setting then.

Problem solved!

I found out how to get FTP to work on my Macs. It was just a hunch, but I tried it and BANGERS ON! (FYI: Bangers is an English slang term for sausages, the kind that is usually enjoyed with a sparkling fermented beverage. Ergo, bangers on implies that the work is done. Time to celebrate.)

As you know, the Macintosh is a very sturdy operating system based almost entirely on Unix, but with a candy apple shell GUI. The developers of this candy coating went to great lengths to provide transparent services to the users that once were only available to the zealous key coders of yesteryear. File Sharing is one such feature. All one has to do is turn it on and specify a folder, and that folder makes itself discoverable on the local network. Oddly though, having it on interferes with passive FTP. All I did to make passive FTP work was to inadvertently decide that maybe … just maybe … it was a problem because it was another kind of File Transfer Protocol. So, I tried turning it off and the rest is history.

BANGERS ON!

Good job!