New Data Upload Method

I’ve been using FTP to transfer my weather data, statistics, and history to the cloppermillweather Web site.  This has worked fairly well, but I didn’t want to be dependent on my ISP providing FTP.  I knew I could create a PHP Web page to accept data.  I thought I might be able to use wget to do an HTTP POST with the data files I wanted to upload.  So I did some researching and discovered that curl was fully capable of doing what I wanted.  So I created a simple PHP page to accept a file and modified my FTP scripts on my weather server to use curl instead of FTP.  Everything is working great with a little less overhead than my prior FTP scripts.  I also added some simple validity checks on the upload page to make sure the data is good.  Not all that secure (at least not yet), but the concept works just fine.  Here’s a generic curl comand to post a file:

curl -F”operation=upload” -F”file=@fileToUpload” http://URLToReceiveHTTPPost

One thing to watch out for is if the destination site sends back an HTTP 301 redirect.  Curl has an option to follow the redirect and do a subsequent POST.  Most browsers will follow the redirect but refuse to do the HTTP POST after the redirect.  This is actually a security feature, so make sure you know where you are being redirected to before POSTing anything you want to keep private.  Weather data probably isn’t something where privacy is a concern, but the warning applies if you are using this technique for other types of data.