Home AMX User Forum AMXForums Archive Threads AMX Applications and Solutions

how do you use IP_CLIENT_OPEN to get info from a website?

13

Comments

  • Here's a test string from the site you're using:
    <yweather:condition  text="Fair"  code="34"  temp="65"  date="Thu, 01 Oct 2009 10:53 am PDT" />
    

    Using Vining's pac man approach, this should also work to remove the temperature using your specific data:
    IF(FIND_STRING(cBuffer,'temp="',1))
      {
    	 REMOVE_STRING(cBuffer,'temp="',1)
    	 nTemp=ATOI(REMOVE_STRING(cBuffer,'"',1)
      }
    
    This should also work in a non-destructive way (don't forget to clear the buffer though when you're done parsing):
    nTempStart	= FIND_STRING(cBuffer,'temp="',1)
    nTempEnd	= FIND_STRING(cBuffer,'date="',nTempstart)
    nTemp		= ATOI(MID_STRING(cBuffer,nTempStart,(nTempEnd-nTempStart)))
    

    The cool thing about the ATOI command is that it tosses out all of the non-numeric stuff for you automatically so you can grab extra (non-numeric) stuff and still be o.k. In the above example
    nTemp = ATOI("temp=65 date=") will come out to be 65. Using this method works out well if you need to pull out a varying length number where you know a character that will act as a delimiter.

    --John
  • Since you're trying to learn, this might be an easier site to work with:
    http://api.wunderground.com/auto/wui/geo/WXCurrentObXML/index.xml?query=89131

    The elements are well separated and easier to parse.

    --John
  • vegastechvegastech Posts: 369
    So let me see if I understand the fundamentals behind the parsing I'm trying to do. I would prefer the non-destructive method, as that seems to sit better with my brain. I'm not used to throwing things out. (read: packrat)
    nTempStart	= FIND_STRING(cBuffer,'temp="',1)
    nTempEnd	        = FIND_STRING(cBuffer,'date="',nTempstart)
    
    
    Find_String locates the string I'm looking for (in this case temp= and date=" ) , which is the beginning and end of the portion of the string I care about.
     nTemp		= ATOI(MID_STRING(cBuffer,nTempStart,(nTempEnd-nTempStart)))
    
    Mid String searches the buffer, starting with the number assigned to nTempStart, ends with what is a mathematical subtraction (in this case, since the digits could be 2 or 3 digits long), and converts my ascii numbers into an integer, and then sets the nTemp variable to the converted temperature number. Am I correct in assuming that mid_string is what you use when you need the ACTUAL characters in the string?

    Is Remove_String the only destructive parsing, out of the string commands? Does this mean that I can continue to re-parse this page for other items in the same manner, without the buffer changing in size?

    Last thing: when it comes to xml parsing, is it better to have these variable assignments occurring within the function call itself, or within the data event's string or offline portions? I tried putting them into the function, but nothing happened with the variables, only the buffer filled:
    DEFINE_FUNCTION fcnWeather() //Weather polling function
    {
    	CLEAR_BUFFER dvIPWeatherBuff
    	IP_CLIENT_OPEN (dvIPWeather.PORT,weather,80,1) //yahoo site
    	IF (find_string(dvIPWeatherBuff, 'temp="',1))
    		{
    		nTempStart = FIND_STRING(dvIPWeatherBuff, 'temp="',1)
    		nTempEnd   = FIND_STRING(dvIPWeatherBuff, 'date="', nTempStart)
    		nTempCurr  = atoi(MID_STRING(dvIPWeatherBuff, nTempStart, (nTempEnd-nTempStart)))
    		SEND_COMMAND dvJVCLT37X898Tp, "'^TXT-10,0, Temperature',itoa(nTempCurr),'F'"
    		}
    }
    


    When I put them within the data_event, everything worked perfectly.
    DATA_EVENT[dvIPWeather]  //data event to query yahoo weather
    {
    	ONLINE:
    	{
    		SEND_STRING dvIPWeather, "'GET http://weather.yahooapis.com/forecastrss?p=89131 HTTP/1.0',13,10, 'Connection: Close', 13, 10, 13, 10"
    				nWeatherOnline = 1
    	}
    	STRING:
    	{
    		
    		dvIPWeatherBuff = data.text
    		SEND_STRING 0, "'STRING dvIPWeatherBuff--------',dvIPWeatherBuff";
    		IF (find_string(dvIPWeatherBuff, 'temp="',1))
    		{
    		nTempStart = FIND_STRING(dvIPWeatherBuff, 'temp="',1)
    		nTempEnd   = FIND_STRING(dvIPWeatherBuff, 'date="', nTempStart)
    		nTempCurr  = atoi(MID_STRING(dvIPWeatherBuff, nTempStart, (nTempEnd-nTempStart)))
    		SEND_COMMAND dvJVCLT37X898Tp, "'^TXT-10,0, Temperature',itoa(nTempCurr),'F'"
    		}
    

    I know that many of you have said to make this run in the function call, however. Any ideas on what's going on here?

    I remember reading about sending the entire string to a file on the controller and then being able to view it - how do I do that, since the telnet window is limited on characters?
  • PhreaKPhreaK Posts: 966
    Yep, you seem to be getting the string parsing down pat. You are correct, REMOVE_STRING is the only destructive function.

    In finding the location of the end of the tempurature string you are interested in you are probably better off doing a search for
    '" '
    
    as this will make sure you'll only be grabbing that attribute if the order of attributes in your feed happens to change.

    The reason you won't be getting anything in your variable when you do your string parsing in the fcnWeather function is that at that point you haven't got anything in your buffer. A good approach would be to break the code that searches for your temperature string into a function of its own, and call that from within the data event. Have a look at the function example I posted on the previous page for an example of how to do this.
  • vegastechvegastech Posts: 369
    What exactly does RETURN do? I see in the netlinx ref. guide it says that it can only be used in a function to return a value if the value is specified in the function, but does RETURN put that data into data.text, or somewhere else, perhaps? I'm not sure what the purpose of using RETURN is.
  • HedbergHedberg Posts: 671
    vegastech wrote: »
    What exactly does RETURN do? I see in the netlinx ref. guide it says that it can only be used in a function to return a value if the value is specified in the function, but does RETURN put that data into data.text, or somewhere else, perhaps? I'm not sure what the purpose of using RETURN is.

    RETURN will terminate a function and will assign a value to the point at which the function is called. You are used to using a function's return value when using the built-in Netlinx functions. For example, in one of the code sections in this thread you will find the ATOI() function:
    nTempCurr  = atoi(MID_STRING(dvIPWeatherBuff, nTempStart, (nTempEnd-nTempStart)))
    

    The ATOI() function returns an integer value and the code assigns that value to the variable nTempCurr. Note also in this example that the return value of the MID_STRING() function is being used as an argument for the ATOI() function.

    When you write a custom function, you can do the same thing. So, the return value of a function gets put where ever you put it.
  • vegastechvegastech Posts: 369
    Ok, that makes sense. So in Phreak's previous example,
    RETURN sValue
    

    is the same as the nTempCurr = at the beginning of my find_string line of code?

    I guess what I'm trying to ask is, since in Phreak's code, it shows:
    sValue = MID_STRING(sString, nStartIndex, nEndIndex - nStartIndex)
        
        RETURN sValue
    
    

    That seems to me like a duplication of the beginning of the MID_STRING example. What basic part am I not understanding? Does RETURN then send sValue to another portion of the code? It seems like netlinx does that on its own.
  • viningvining Posts: 4,368
    In Phreak's example calling the function:
    Code:
    nTemp = ATOI(fnGetXMLAttributeValue(sBuffer, "'temp'")
    

    He's calling his function fnGetXMLAttributeValue(sBuffer, "'temp'") and passing to this function a string (sBuffer) & the word or data he wants to find, in this case "temp". In the function they assume these names in the same order they are passed in (CHAR sString[], CHAR sAttribute[]). Now the function does its stuff & the result is sValue which he "returns" to the code that called the function.

    Now the code that called the function nTemp equals sValue but first the ATOI function is performed on it to convert it to an integer. Essentially you have nTemp = ATOI(sValue) since sValue is the return of the function.
  • vegastechvegastech Posts: 369
    But doesn't the code already return the results, which are then put into the variable sValue? Why the need to use the keyword RETURN? Is this something that has to be done when creating local variables within a function?
  • vegastechvegastech Posts: 369
    Ok, well it really doesn't matter at this point, as it's working. Cool with me. Final Question, I hope: Is there an easier way to do what I'm doing here, as far as stacking functions/If statements? Currently my parsing consists of a function for each weather item, like so:
    DEFINE_FUNCTION fcnWeatherVis()
    {
    	STACK_VAR nVisStart
    	STACK_VAR nVisEnd
    	STACK_VAR nVisCurr
    	IF (find_string(dvIPWeatherBuff, 'visibility="',1))
    	{
    		
    		
    		nVisStart  = FIND_STRING(dvIPWeatherBuff, 'visibility="',1)
    		nVisEnd   = FIND_STRING(dvIPWeatherBuff, '" ', nVisStart)
    		nVisCurr  = atoi(MID_STRING(dvIPWeatherBuff, nVisStart, (nVisEnd-nVisStart)))
    		 
    		SEND_COMMAND dvJVCLT37X898Tp, "'^TXT-30,0, Visibility ',itoa(nVisCurr),' Mi.'"
    	}
    }
    

    I thought about doing a Switch...Case, but that didn't work since I need ALL the items to return results. I looked at an IF, ELSE IF, ELSE IF, ELSE statement, but I would still be doing the same amount of work, where I need multiple variables of different names for each weather item. I'm not sure what my options are for this.
  • viningvining Posts: 4,368
    If you use global variables to hold your values then you might not need to return a value but often we like to limit the scope of our variables and make the working variable only have scope (accessability & purpose) inside the code that actually uses it.

    Well you could make an array of "attributes" or "strings" you want to look for and use that in a loop with the function call in it or just do a bunch of "if's" like in the examples posted earlier.
  • vegastechvegastech Posts: 369
    Ok, so other than a multi-dimensional array (I think that's what we are talking about here), it's pretty much however I feel like writing it (copy and paste isn't too bad right about now! ;) )

    One of the things I am trying to do is to pull the actual date and time info from the XML file. I thought a function like this would work:
    DEFINE_FUNCTION fcnWeatherDateTime()
    {
    	STACK_VAR nDateStart
    	STACK_VAR nDateEnd
    	STACK_VAR nDateCurr
    	STACK_VAR cDateCurr
    	IF (find_string(dvIPWeatherBuff, '<lastBuildDate>',1))
    	{
    		
    		
    		nDateStart  = FIND_STRING(dvIPWeatherBuff, '<lastBuildDate>',1)
    		nDateEnd   = FIND_STRING(dvIPWeatherBuff, '</lastBuildDate>', nDateStart)
    		sDateCurr  = MID_STRING(dvIPWeatherBuff, nDateStart, (nDateEnd-nDateStart))
    		
    		SEND_COMMAND dvJVCLT37X898Tp, "'^TXT-22,0,',cDateCurr"
    	}
    }
    

    But the compiler is telling me
    WARNING: Main.axs(315): C10570: Converting a string to a [INTEGER]  
    

    when I try to pull the actual ascii. I don't want to convert it - just send it as-is. Can I do that?
  • viningvining Posts: 4,368
    Sure, just change your data type on the stack var:

    STACK_VAR cDateCurr
    to
    STACK_VAR CHAR cDateCurr[10] ; //what ever length is required!

    w/o designating the data type as a "char" type it's default is an integer. In your code you're getting an error because you're trying to put a string into an integer.
  • PhreaKPhreaK Posts: 966
    vegastech wrote: »
    But doesn't the code already return the results, which are then put into the variable sValue? Why the need to use the keyword RETURN? Is this something that has to be done when creating local variables within a function?

    It is possible to program solely with global variables and just have everything updating them from all over your code. Although this may appear easier at first, as your code grows you'll find your namespace becoming increasingly cluttered and it will become extremely hard to track where your variables are being updated from, making debugging a right pain in the arse. Your code will also become much harder to re-use and other developers working on it in the future may feel the urge to hit your with a wet fish as they will have to mentally map out the entire program rather than being able to modify and streamline discrete sections of code which have been designed to perform a specific function.

    Thankfully though you can escape this assult by seafood and avoid tearing your hair out when you're debugging by utilizing encapsulation. Although NetLinx doesn't provide the level of encapsulation available in a fully fledged object orientated language you can still encapsulte a lot of functionality both on a higher level, using modules (and pseudo encapsulation via includes), and on a lower level with functions. Functions (and modules) allow you to not only encapsulate blocks of related logic, but also any variables that are required to perform that logic.

    Using the return keyword in a function along with parameters which you pass to it allows the internal logic to be completely oblivious to the outside word. It is a section of code designed to do a single task as effeciently as possible.

    That being said, a function may also have other side effects such as communicating with a device, writing or reading a file, or updating a global variable. However, whenever possible you will find life will be much easier if you encapsulate as much as possible.
  • In finding the location of the end of the tempurature string you are interested in you are probably better off doing a search for
    Code:

    '" '

    as this will make sure you'll only be grabbing that attribute if the order of attributes in your feed happens to change.
    I agree too. In my example
    nTempStart	= FIND_STRING(cBuffer,'temp="',1)
    nTempEnd	= FIND_STRING(cBuffer,'date="',nTempstart)
    nTemp		= ATOI(MID_STRING(cBuffer,nTempStart,(nTempEnd-nTempStart)))
    

    I was showing off how ATOI drops the non-numeric characters from the result. Definitely better in most cases to parse precisely.

    --John
  • I wrote a quick sample parsing routine below using a FOR loop that works with most of the elements from the yahoo site (which I don't like so much) you're using. It organizes the search elements in one array so that they're easy to edit.

    This should give you an idea of one way to modularize your weather program. I always enjoy seeing how others approach their coding problems, maybe others will post some more sample code for you too.

    PROGRAM_NAME='Weather Parse Test'
    (***********************************************************)
    (*  FILE CREATED ON: 10/05/2009  AT: 09:03:59              *)
    (***********************************************************)
    (***********************************************************)
    (***********************************************************)
    (*  FILE_LAST_MODIFIED_ON: 10/05/2009  AT: 09:16:34        *)
    (***********************************************************)
    (* System Type : NetLinx                                   *)
    (***********************************************************)
    (* REV HISTORY:                                            *)
    (***********************************************************)
    (*
        $History: $
    *)
    (***********************************************************)
    (*          DEVICE NUMBER DEFINITIONS GO BELOW             *)
    (***********************************************************)
    
    DEFINE_DEVICE dvIPWeather = 0:2:0
    
    (***********************************************************)
    (*               CONSTANT DEFINITIONS GO BELOW             *)
    (***********************************************************)
    DEFINE_CONSTANT
    CHAR cRSS_EOF[]		      =   '-->'
    
    CHAR sElementsToParse[16][30] =  {'location city="',			'"',
    				  'Current Conditions:</b><br />',	'<',
    				  'temp="',				'"',
    				  'units temperature="',	        '"',
    				  'visibility="',			'"',
    				  'distance="',			        '"',
    				  'sunrise="',			        '"',
    				  'sunset="',			        '"'
    										  }
    
    
    (***********************************************************)
    (*              DATA TYPE DEFINITIONS GO BELOW             *)
    (***********************************************************)
    DEFINE_TYPE
    
    (***********************************************************)
    (*               VARIABLE DEFINITIONS GO BELOW             *)
    (***********************************************************)
    DEFINE_VARIABLE
    CHAR sWeatherElements[15][60] //City
    										//CurrentConditions
    										//Temperature
    										//Temp UOM
    										//Visibility
    										//Visibility UOM
    										//Sunrise
    										//Sunset
    CHAR cBuffer[3250]								  
    INTEGER nNumberOfAttributes
    (***********************************************************)
    (*               LATCHING DEFINITIONS GO BELOW             *)
    (***********************************************************)
    DEFINE_LATCHING
    
    (***********************************************************)
    (*       MUTUALLY EXCLUSIVE DEFINITIONS GO BELOW           *)
    (***********************************************************)
    DEFINE_MUTUALLY_EXCLUSIVE
    
    (***********************************************************)
    (*        SUBROUTINE/FUNCTION DEFINITIONS GO BELOW         *)
    (***********************************************************)
    (* EXAMPLE: DEFINE_FUNCTION <RETURN_TYPE> <NAME> (<PARAMETERS>) *)
    (* EXAMPLE: DEFINE_CALL '<NAME>' (<PARAMETERS>) *)
    DEFINE_FUNCTION CHAR[65] fnWeatherAttrParse(CHAR cAttribute[],CHAR cEndChar[], CHAR cString[])
    {
      STACK_VAR INTEGER nAttributeStart
      STACK_VAR INTEGER nAttributeEnd
      STACK_VAR CHAR 	  sValue[65]
      nAttributeStart = (FIND_STRING(cString,"cAttribute",1))
      IF(nAttributeStart)
      {
    	 nAttributeStart	= nAttributeStart+LENGTH_STRING(cAttribute)
    	 nAttributeEnd		= (FIND_STRING(cString,"cEndChar",nAttributeStart))
    	 sValue				= (MID_STRING(cString,nAttributeStart,nAttributeEnd-nAttributeStart))
      }
      ELSE
    	 sValue = '-'
    	 
      RETURN sValue
    }
    
    (***********************************************************)
    (*                STARTUP CODE GOES BELOW                  *)
    (***********************************************************)
    DEFINE_START
    nNumberOfAttributes = (LENGTH_ARRAY(sElementsToParse))/2
    
    
    (***********************************************************)
    (*                THE EVENTS GO BELOW                      *)
    (***********************************************************)
    DEFINE_EVENT
    DATA_EVENT[dvIPWeather]
    {
      STRING:
      {
    	 LOCAL_VAR INTEGER nParseLoop
    	 cBuffer="cBuffer,DATA.TEXT"
    	 IF(FIND_STRING(cBuffer,"cRSS_EOF",1))
    	 {
    		FOR(nParseLoop=1;nParseLoop<=nNumberOfAttributes;nParseLoop++)
    		{
    		  sWeatherElements[nParseLoop] = fnWeatherAttrParse(sElementsToParse[(((nParseLoop-1)*2)+1)],sElementsToParse[(((nParseLoop-1)*2)+2)],cBuffer)
    		}
    	 REMOVE_STRING(cBuffer,"cRSS_EOF",1)
    	 }
    	 
      }
      
    }
    (***********************************************************)
    (*            THE ACTUAL PROGRAM GOES BELOW                *)
    (***********************************************************)
    DEFINE_PROGRAM
    
    (***********************************************************)
    (*                     END OF PROGRAM                      *)
    (*        DO NOT PUT ANY CODE BELOW THIS COMMENT           *)
    (***********************************************************)
    
  • Vining also has a great example of dealing with RSS feeds, and IP data in his weather module that he shared with everyone here. Here's the module itself:
    http://www.amxforums.com/showpost.php?p=17037&postcount=18

    --John
  • Joe HebertJoe Hebert Posts: 2,159
    vining wrote:
    In Phreak's example calling the function:
    vining wrote:
    He's calling his function fnGetXMLAttributeValue...

    He? His? I believe those pronouns should be filtered through a bitwise not.
  • PhreaKPhreaK Posts: 966
    Joe Hebert wrote: »
    He? His? I believe those pronouns should be filtered through a bitwise not.

    No 'He' is correct. Unless there's something my parents didn't tell me.
  • Joe HebertJoe Hebert Posts: 2,159
    PhreaK wrote: »
    No 'He' is correct. Unless there's something my parents didn't tell me.

    Oops. Color me embarrassed. Sorry.

    I assumed female after reading this reply a while back.

    http://amxforums.com/showpost.php?p=33859&postcount=26
  • PhreaKPhreaK Posts: 966
    Ha ha ha. Yeah, maybe I should have put a note on that.
  • vegastechvegastech Posts: 369
    Ok, now it's REALLY starting to get interesting..How would a multi-dimensional array be more useful in a setup like John describes, than just a single-dimension array? Is there a way to search and find items in rows as well as the standard column the reference guide shows?

    John, how is it that the cAttribute item is filled with data? I see it referenced here:
    DEFINE_FUNCTION CHAR[65] fnWeatherAttrParse(CHAR cAttribute[],CHAR cEndChar[], CHAR cString[])
    {
      STACK_VAR INTEGER nAttributeStart
      STACK_VAR INTEGER nAttributeEnd
      STACK_VAR CHAR 	  sValue[65]
      nAttributeStart = (FIND_STRING(cString,"cAttribute",1))
    

    but nothing prior to the above makes any reference to cAttribute. I don't see anything in the data event either. What is the purpose of setting the Att, End, and String variables as parameters in this function? I'm only used to defining a function without any parameters. Sorry about all the questions.
  • PhreaKPhreaK Posts: 966
    cAttribute, cEndChar and cString are what are called parameters. These are values that are passed to the function when you call it and allow you to feed data into a function for processing. This is part of what I was talking about in my earlier post about encapsulation. To help understand it better if you were to call fnWeatherAttrParse like so:
    fnWeatherAttrParse("'an_attribute'", "'end character'", "'this is the string which will be parsed'")
    
    When fnWeatherAttrParse() runs it will have 3 variables with a local scope (only visable to code within the the braces of fnWeatherAttrParse)
    cAttribute == 'an_attribute'
    cEndChar == 'end character'
    and cString == 'this is the string which will be parsed'
    
    If you were to then call fnWeatherAttrParse as
    fnWeatherAttrParse("'another_attribute'", "'another end character'", "'this is a different string")
    
    When the code inside fnWeatherAttrParse runs this time the three variables created by the parameters will be:
    cAttribute == 'another_attribute'
    cEndChar == 'another end character'
    and cString == 'this is a different string
    
    This allows your one block of code to process different bits of data without the need to repeat your logic or put values into global variables. It basically allows less code to do more.
  • a_riot42a_riot42 Posts: 1,624
    PhreaK wrote: »
    cAttribute, cEndChar and cString are what are called parameters. These are values that are passed to the function when you call it and allow you to feed data into a function for processing.

    I thought the values that were passed to functions were called arguments.
    Paul
  • PhreaKPhreaK Posts: 966
    As far as my knowledge goes cAttribute, cEndChar and cString are parameters. The data passed into these when the function is called are arguments. I could however be completely wrong.
  • Joe HebertJoe Hebert Posts: 2,159
    PhreaK wrote: »
    As far as my knowledge goes cAttribute, cEndChar and cString are parameters. The data passed into these when the function is called are arguments.
    That?s my understanding also.
    The parameter list is part of the definition.
    The arguments are the things passed in.
  • HedbergHedberg Posts: 671
    PhreaK wrote: »
    As far as my knowledge goes cAttribute, cEndChar and cString are parameters. The data passed into these when the function is called are arguments. I could however be completely wrong.

    Oh, boy, am I in trouble. I'm one of those people who has used the terms pretty much interchangeably. Now I'm going to have to go back and rewrite all my code.
  • Joe HebertJoe Hebert Posts: 2,159
    Hedberg wrote: »
    Oh, boy, am I in trouble. I'm one of those people who has used the terms pretty much interchangeably. Now I'm going to have to go back and rewrite all my code.
    I?ve been scanning the airwaves and I heard that the lingo police have a warrant out for your arrest.
    Consider yourself warned.
  • HedbergHedberg Posts: 671
    Joe Hebert wrote: »
    I?ve been scanning the airwaves and I heard that the lingo police have a warrant out for your arrest.
    Consider yourself warned.

    Yeah, don't want to get into a parameter with the lingo police.
  • Hedberg wrote: »
    Yeah, don't want to get into a parameter with the lingo police.

    hahahaha killing me! That was good.

    --John
Sign In or Register to comment.