Home› AMX User Forum› NetLinx Studio

Parsing Data from a Webpage

2»

Comments

  • jweatherjweather Posts: 320
    Actually, http://inteliplex.com/wx and http://inteliplex.com/wx/ are two different URLs. You're requesting the former, it's redirecting you to the latter. Send GET /wx/ HTTP/1.1 and all should be well.

    I believe it does the redirect so that if you have relative links on the /wx/index.html page (say to test.html), they redirect consistently to /wx/test.html in all cases rather than to /test.html if the user browsed to http://inteliplex.com/wx -- make sense?

    Jeremy
  • mpullinmpullin Posts: 949
    jweather wrote: »
    Actually, http://inteliplex.com/wx and http://inteliplex.com/wx/ are two different URLs. You're requesting the former, it's redirecting you to the latter. Send GET /wx/ HTTP/1.1 and all should be well.

    I believe it does the redirect so that if you have relative links on the /wx/index.html page (say to test.html), they redirect consistently to /wx/test.html in all cases rather than to /test.html if the user browsed to http://inteliplex.com/wx -- make sense?

    Jeremy
    Interesting... thanks for pointing that out.
  • jjamesjjames Posts: 2,908
    When I send this (ZIP_CODE is 65065) 
    SEND_STRING 0:nLOCAL_PORT+1:0,"'GET /weather/map/',ZIP_CODE,' HTTP/1.1',13,10,
	'Host: www.weather.com',13,10,
	'User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0',13,10,
	'Accept: */*',13,10,
	'Accept-Language: en-us,en;q=0.5',13,10,13,10"

    I get this . . . 
    HTTP/1.1 307 Temporary Redirect
Date: Fri, 20 Jun 2008 20:37:58 GMT
Server: Apache
Set-Cookie: partner=www; domain=(null); path=/; Version="1";
Set-Cookie: partner_url=(null); domain=(null); path=/; Version="1";
Location: [url]http://www.weather.com/weather/map/65065[/url]
Content-Length: 443
Keep-Alive: timeout=3, max=1
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>307 Temporary Redirect</title>
</head><body>
<h1>Temporary Redirect</h1>
<p>The document has moved <a href="http://www.weather.com/weather/map/65065">here</a>.</p>
<p>Additionally, a 307 Temporary Redirect
error was encountered while trying to use an ErrorDocument to handle the request.</p>
<hr>
<address>Apache Server at [url]www.weather.com[/url] Port 80</address>
</body></html>

    Anyone? Please? :D
  • slip couganslip cougan Posts: 34
    i!-EquipmentMonitorOut.axi

    Where can I find this file?

    Thanks

    gary
  • viningvining Posts: 4,368
    Try this URL: 
    http://www.weather.com/weather/tenday/USMO0665?cm_ven=LWO&cm_pla=city_page&cm_ite=10-day&site=city_page&cm_cat=rss&locid=USMO0665&par=LWO_rss
  • jjamesjjames Posts: 2,908
    Thanks, but I don't need the forecast, I'm trying to get the "local" radar for a specified zip code, and trying to scrape the image from returned html. :(
  • jjamesjjames Posts: 2,908
    slip cougan wrote: »
    Where can I find this file?
    Gary,
    Log into AMX.com, then go to Tech Center -> Application Files -> integration! solutions and it should be there.
  • viningvining Posts: 4,368
    OK, try this site.

    http://www.srh.noaa.gov/ridge/lite/


    Click on a radar image type and then select an area from the list. Here's some snipets of code to help you figure out what your looking at. 
    char Rad_Image_Type [9][3]=

     {
     'N0R', // Base Reflectivity
     'N0S', // Storm Relative Motion
     'N0V', // Base Velocity
     'N1P', // One-Hour Precipitation
     'NCR', // Composite Reflectivity
     'NTP', // Storm Total Precipitation
     'N0Z', //Base Reflectivity //Long Range image (view out to 248 nmi)
     'NET', //
     'NVL'  //
     } 

 //MISSOURI 
     RadState.Missouri[1].Area = 'Kansas City' ;
     RadState.Missouri[2].Area = 'Springfield' ;
     RadState.Missouri[3].Area = 'St. Louis' ; 
     RadState.Missouri[1].Code = 'eax' ;  
     RadState.Missouri[2].Code = 'sgf' ;  
     RadState.Missouri[3].Code = 'lsx' ;

    I never quite finished this module and I intended on making every radar site in the US available. I did finish the Satellite portion, I think?
  • jjamesjjames Posts: 2,908
    Thanks . . . I think I'm being too picky. LOL!

    I can go ahead and scrape out the image I need if I can get to the source I see in my browser, I just think there's too much redirection going on. I want to stick with Weather.com since I think they're the most familiar and legible radar maps for the common client.

    Thanks though!!!
  • jjamesjjames Posts: 2,908
    Quick update: I was finally able to find it. I had to include the "encoding" attribute, and I was then able to read the entire page and get the data I needed.
  • Chip MoodyChip Moody Posts: 727
    What a PIA! Glad ya' found it though.
    jjames wrote: »
    I had to include the "encoding" attribute
  • jjamesjjames Posts: 2,908
    Chip Moody wrote: »
    What a PIA! Glad ya' found it though.

    Thanks . . . actually, let me show you what I had to send: 
    SEND_STRING 0:nLOCAL_PORT+1:0,"'GET /forecast/',ZIP_CODE,'.html HTTP/1.1',13,10,
			'Host: weather.yahoo.com',13,10,
			'User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)',13,10,
			'Accept-Encoding: deflate;q=0',13,10,
			'TE: deflate;q=0',13,10,
			'expires: #now()#',13,10,
			'Keep-Alive: 300',13,10,
			'Cache-Control: no-cache, no-store, must-revalidate',13,10,
			'charset: utf-8',13,10,13,10"

    It wasn't the "encoding" (my bad), it was the "charset". After a while Googl'ing, I came across this suggestion and it sure enough worked. It was a PITA . . . I've got it all working now, except I cannot load the Alaskan Doppler radar becase, well . . . there isn't one. [EDIT: There isn't a 600 mile radar, only the 2100 - though that might be good enough.]

    So, now I need to adjust it . . . ugh. Oh well, the stuff you gotta go through to do something "cool" and probably won't be used much. ;)
Sign In or Register to comment.