Home NetLinx Studio

Serial Data Parsing Question on Projector video wall

JohnMichnrJohnMichnr Junior MemberPosts: 276
Hey All - I'm looking for help on a video wall that I am programming. I have 24 cube wall that I want to parse status on during power on & power off. I have a timeline sending out the status command to each wall for the duration of the poweron/off cycle ( I shut the status command off for the cubes that are either full on or full off) Then once a wall reports full on I ask for lamp 1 hours, the responce to that triggers the request for lamp 2 hours. At first everything seemed like it was working.... (like starting a novel with It was a dark and stormey night...)

But - I my program started locking up and the system was throwing out diagnostics error messages GetString - Error1 and CopyString Error-1. As I was looking into that I noticed that the wall would sometimes stop responding for 4-5 seconds - then send out a tons of responces, and some garbage thrown in there just for fun. (It responded with a power responce for a cube that was not there)

Now the problem that I am having is that the responce data has a start byte, no end byte, and is differing lengths (either 7 or 9 bytes long) I have to make sure that I have full and correct responce byte before hitting my parsing. And the start byte is $02 that can be used anywhere in the command - not just the start byte.

Here is a typical responce: $02,$04,$A0,$01,$00,$00,CS,$01
First $02 - Start Byte
$04 - data length
$A0 - Command (this case status)
$01,$00,$00 - three bytes of data (plust the command above equals the $04 for data length)
CS - check sum two's complement of length command and data (previous post)
$01 - what projector is responding.

Here is what I am using:
      stack_var char sMsg[9]
      stack_var integer nWhich
      stack_var char nLamp
      stack_var char nTempLamp[3]
      stack_var integer nSize
      stack_var char nCmd
      stack_var integer N
      stack_var integer nLocation
     sWallRxbuf = "sWallRxbuf,data.text"
     while(find_string(sWallRxbuf,"$02,$04,$A0",1) || find_string   (sWallRxbuf,"$02,$05,$A1",1) 
     || (LENGTH_STRING(sWallRxbuf) > 8))//lets parse this
	sTrash = remove_string(sWallRxbuf,"$02",1)
	nSize = get_buffer_char(sWallRxbuf)+2
  	      sWallRxbuf = "$02,nSize-2,sWallRxbuf" //put back what you just took out
	    sMsg = GET_BUFFER_STRING(sWallRxbuf,nSize)
	    nCmd = GET_BUFFER_CHAR(sMsg)
	    switch (nCmd)
	        case $A0: //status command
		cBit[1] = sMsg[1] & 1
		cbit[2] = sMsg[1] & 2
		cbit[3] = sMsg[1] & 4
		cbit[4] = sMsg[1] & 8
		cbit[5] = sMsg[1] & 16
		cbit[6] = sMsg[1] & 32
		cbit[7] = sMsg[2] & 1
		cbit[8] = sMsg[2] & 2
		cbit[9] = sMsg[2] & 4
		cbit[10] = sMsg[2] & 8
		cbit[11] = sMsg[2] & 16
		cbit[12] = sMsg[2] & 32
		cbit[13] = sMsg[2] & 64
		cbit[14] = sMsg[3] & 1
		cbit[15] = sMsg[3] & 2
		cbit[16] = sMsg[3] & 16
		cbit[17] = sMsg[3] & 32
		cbit[18] = sMsg[3] & 128
		sTrash = get_buffer_string(sMsg,4)
		nWhich = get_buffer_char(sMsg)
		       cCubeStatus[nWhich] = N-1
		     call 'CUBE STATUS' (nWhich)
		switch (cCubeStatus[nWhich])
		    case 4: //ERROR ON CUBE
		       cCubeError[nWhich] = ''
		          cCubeError[nWhich] = "cCubeError[nWhich],';',nCubeErrorTxt[n]"
		    case 0: //cube is off
		    case 5: //cube is on
			call 'WALL XMIT' (nGetLamp1,nWhich)
	case $A1: //lamp hours status
	      nLamp = get_buffer_char(sMsg)
	      nTempLamp = get_buffer_string(sMsg,3)
	      sTRASH = get_buffer_char(sMsg)
	      nWhich = GET_BUFFER_CHAR(sMsg)
	      switch (nLamp)
		case $20: //lamp 1
	  	    cCubeLampHours[nWhich][1] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]
		    call 'WALL XMIT' (nGetLamp2,nWhich)
		case $40: //Lamp 2
		       cCubeLampHours[nWhich][2] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]

This code work fine to parse teh data, but only when I have a full and complete string going into it.

I need some help getting the data into the while loop. I think I need to create a loop that grabs each byte one at a time (get_buffer_char) then if the byte is a $02 - look 2 byte ahead and if it is a $A0 or $A1 (the only 2 I am working on) go back one byte to the data length and get that many bytes plus 2 (check sum and which projector)

But I am not that sure how to do it. Any suggestions? Any other thoughts?

Thanks in advance...


  • NMarkRobertsNMarkRoberts Junior Member Posts: 455
    Here is my pseudocode of how I would do it:

    If $02 is present
    and next (data length) byte is present
    Throw away everything before the $02
    if at least the correct amount of data ($02 + 1 + data length + 2) is present
    Cut the string out of the incoming stream leaving the rest for later
    if the checksum is correct (call a separate subroutine)
    and the projector number is sensible
    Parse and interpret the comand
  • DHawthorneDHawthorne Junior Member Posts: 4,584
    I hate dealing with serial packets with no distinct terminaton, and especially with variable lengths. But, they are out there, and have to be dealt with.

    I buffer the entire response, and peel off char by char. Generally, right after the start char is some indication of how big the packet is going to be: either a response code that has a specific data length to follow, a byte telling you how much data is to follow, or a combination of those. With that, you can build up your response in a holding variable, and finish it up when you have all the data you are expecting. If you get a new start character before the packet finished, throw it away (and perhaps send your debug window a message in case there is something wrong and not just junk data).

    My experience is that equipment that uses this kind of protocol tends to be working at a fairly low level on the serial port; it's not bufered or interpreted within the device, and therefore more prone to errors on the output. So you have to be more careful with your error handling and make sure your code will deal with it gracefully rather than barfing all over your design.
  • JohnMichnrJohnMichnr Junior Member Posts: 276
    Yeah Great - you got any code samples that I could look at for teh character by character buffer? I figure I need a while statement in the data parsing somewhere - incase there is more than one command in the buffer (or a timeline just running to parse the data) not sure which way to go.
  • DHawthorneDHawthorne Junior Member Posts: 4,584
    Basically, it's a matter of taking the code out of your STRING handler in the data event. Make a buffer of suitable size to hold everything that might come in at once, and put a CREAT_BUFFER statement in your DEFINE_START section. The master itself will now take every bit of feedback on that port, and append it to that buffer variable. Lets say you defined it as CHAR sVidWallBuffer[1024]. In DEFINE_START, put CREATE_BUFFER dvVidWall, sVIdWallBuffer. Then in DEFINE_PROGRAM, put a line that says something like IF(LENGHT_STRING(sVidWallBuffer)) doParseVidWall (or whatever you wish to name your parsing routine).

    Your doParseVidWall function needs to have a local variable of the proper size for a single packet. What it needs to do is take the highest order character off the buffer, and append it to this local variable. For triggering purposes, I would also make a single char for holding each chunk so you can test it and decide what to do. You could do something like:
    DEFINE_FUNCTION doParseVidWall()
    STACK_VAR CHAR cDigit[1]
    LOCAL_VAR CHAR sPacket[256]
    cDigit =  LEFT_STRING(sVidWallBuffer, 1)  // get first character
    sVidWallBuffer = RIGHT_STRING(sVidWallBuffer, LENGTH_STRING(sVidWallBuffer) - 1) // remove character from buffer
    IF(cDigit == "$02)
         CLEAR_BUFFER sPacket // we have a start char, clear data and start over
         sPacket = "sPacket, cDigit"    //  append new data to the packet variable
         IF(LENGTH_STRING(sPacket) == <amount of data we are expecting>
             <code to process packet>

    I'm sure there are other ways, that's just how I've handled similar situations. It would seem to me though, that the bulk of your problem is the STRING handler. What determines when that will fire on a NetLinx is when the master sees a pause in the data stream, it assumes the data is finished. So you are very likely to get incomplete data bursts using it if anything at all, including normal serial port delays in the equipment, makes that data pause. It will be in the next burst, it's not lost, but that is likely to throw off your parsing. Buffering it and testing char by char eliminates this. In my opinion, the STRING handler on a port is only suitable when you know for certain you will always get a full and complete packet of data every time - which is almost never with a serial port. I personally always buffer serial ports, and only use that handler with intra-master virtual messages.
  • Joe HebertJoe Hebert Junior Member Posts: 2,154

    This is the approach I?ve used in the past. It?s similar to what NMarkRoberts eluded to earlier. Here is some code:
    dvProj	= 5001:1:0
    INTEGER nMinLength = 7
    INTEGER nCmdPad	  = 3 //the $02 + checksum byte,+ proj# byte
    CHAR	cBuffer[1024]
    DEFINE_FUNCTION fnParseBuffer () {
       CHAR	    junk[64]
       INTEGER junksize
       INTEGER cmdsize
       CHAR	    cmd[64]
       //do we at least have the minimum requirements for a valid message?
       IF (LENGTH_ARRAY(cBuffer)>=nMinLength && FIND_STRING(cBuffer,"$02",1)) {
          //lets check to see if any garbage is in fromt
          junksize = FIND_STRING(cBuffer,"$02",1) - 1
          IF (junksize) {
    	 //strip the garbage out
    	 junk = GET_BUFFER_STRING(cBuffer,junksize)
    	 SEND_STRING 0, "'**** JUNK - ',ITOA(junksize),'junk characters getting tossed - ',junk"
    	 //lets spin out if the junk reducted our length to less than minimum
    	 IF (LENGTH_ARRAY(cBuffer) < nMinLength) RETURN
          //we're still here so lets find out how long this command is
          cmdsize = cBuffer[2] + nCmdPad
          IF (LENGTH_ARRAY(cBuffer)>=cmdsize) { //do we have the entire command here?
    	 //we do so lets strip it
    	 cmd = GET_BUFFER_STRING(cBuffer,cmdsize)
    	 IF (fnCheckChecksum(cmd)) { //if checksum is valid
    	    fnProcessCommand(cmd)  //pass this command on to be processed.
    	 ELSE {
    	    SEND_STRING 0, "'Invalid checksum received with cmd: ',cmd"
          ELSE {
    	 SEND_STRING 0, "'Don''t have entire response yet'"
    	 //time to spin out and try again next go around
    DEFINE_FUNCTION INTEGER fnCheckChecksum (char incoming[]) {
       //do whatever to check the checksum
       RETURN 1 //if checksum is good
    DEFINE_FUNCTION fnProcessCommand (char cmd[]) {
       //see if it's a command you want to do something with and do it
    CREATE_BUFFER dvProj,cBuffer
    //I usually have this in mainline.
    //It's probably more correct to put it in a TIMELINE
    //Even more correcter (huh?) I think would be to put 
    //this in the STRING: handler for dvProj but feel I
    //might miss something if I do. I'm curious where others put it
    (***************THE END****************)

  • DHawthorneDHawthorne Junior Member Posts: 4,584
    Thanks for the more complete example Joe, I just can't get my brain going fast enough first thing in the morning to post full code blocks like that :) ... and the rest of the day, I'm posting bewtween compiles and reloads if I'm on at all.

    But in answer to your question in the commented out portions: I would put it in mainline too. For all the reasons mentioned above, I don't think the STRING section is quite reliable for every possible situation where it might get only partial data. Perhaps it's just paranoia.
  • JohnMichnrJohnMichnr Junior Member Posts: 276
    Wow - Dudes... Thanks for the responce - I'm at a site with 4 of the cubes staring me in the face right now, and I'll try these to see what happens. I kind of like using the string handler for this stuff, although Dave I understand the reasons to try it elsewhere.

    Again Thanks!

    Joe I'll name a comemorative sub routine for you in my program...
  • Joe HebertJoe Hebert Junior Member Posts: 2,154
    JohnMichnr wrote:
    Joe I'll name a comemorative sub routine for you in my program...
    Can you make it a commemorative function instead? :D

    Let us know how the story ends.
  • JohnMichnrJohnMichnr Junior Member Posts: 276
    Can you make it a commemorative function instead?

    I knew you were going to say that...

    Actually I spent 6 quality hours with a 4 cube version on the wall yesterday with mixed results. The code that Joe sent out (thanks again) worked mostly, a few mode here and there and it seemed to come together. The big problem is that the wall goes stupid 5 seconds in and then waits 5 seconds and spits out a ton of stuff, some of it good, some of it not so good. The other thing the wall does is put out an ACK responce to some commands, any command going to cube #2, of $10,$02 like the same $02 that is supposed to be a start code for the responces... So the buffer gets one character off because it has 2 $02 in a row. But I can remove that just by looking for that ACK responce and deleting it before it gets to the buffer

    The real issue now is that the Hebert Code (TM) and my parsing code is throwing out a bunch of GetString error 1 and CopyString Error 1 responces in diagnositcs. I don't know if that is cuaseing the other big error...
    The customer did not want to purchase a touchpanel so we are using a G3 web panel for the control. That panel keeps locking up when I fire up the video wall as a whole. I can fire up individual cubes using another page and not have it lock up, but firing up the wall as a whole locks up the control panel, Sometimes. The only difference on the whole wall firing is that I add 2 more timelines going for a countdown timer on the webpanel and a general timer for the startup sequence of the system. So I don't know if that is the problem, or what. I don't have anymore time infront of the wall until I go onsite next week, but I programmed another cahssis to be a video wall emulator (except for the 5 second stupid period - but I am considering adding that) so I will be testing with my emulator until then.

    The funny thing with this is - Ive been working so hard to get teh status to come up on a touchpanel page - I never really wondered if I really needed it. The wall consistan'y turns on & off with teh commands, I have never had that fail. Just problems with the status parse. So I am thinking of backing down my requirement a little.
  • Joe HebertJoe Hebert Junior Member Posts: 2,154
    John wrote:
    code is throwing out a bunch of GetString error 1 and CopyString Error 1 responces in diagnositcs.
    If you?re not already, compile with Debug Info and you will get a line numbers with the runtime errors which should be helpful in tracking down the culprit.

    There shouldn?t be any cases where the code I posted (unmodified) will cause any runtime errors.
    John wrote:
    The funny thing with this is - Ive been working so hard to get teh status to come up on a touchpanel page - I never really wondered if I really needed it.
    Been there done that. :)
  • JohnMichnrJohnMichnr Junior Member Posts: 276
    Well - I have just finished purging the Get String/Copy String errors from my program. I had help from Tech Support but it amounted to the following lines.

    By the way - the debug info did not show any line numbers for these issues - don't know why

    From Joes original code above ( And I did it as well a bunch of times)
    nCmdPad = 3
    CHAR cBuffer[1024]
    DEFINE_FUNCTION fnParseBuffer () {
       CHAR	    junk[64]
       INTEGER   junksize
       INTEGER   cmdsize
       CHAR	    cmd[64]
    cmdsize = cBuffer[2] + nCmdPad

    cmdsize is an integer, and 16 bits, 1 byte of cBuffer is a char and 8 bits. Relating an 8 bit byte to the 16 bit integer would through up those errors - as below
    Line 25 :: GetString - Error 1 Tk=0x0001 - 16:01:34
    Line 26 :: CopyString (Reference) - Error 1 S=0x0000 D=0x1011 - 16:01:34
    from the Diagnostic window.

    tech support wanted me to BAND the single 8 bit byte with $FF before equating it to the integer. as below
    cmdsize = (cBuffer[2] & $FF) + nCmdPad
    going through and removing all of those in Joe's code and my own code got rid of all the errors.

    The part I don't get is that the first code worked - even though it through out the error, and I know that I have done that routine before (equating a single portion of a character array to an integer variable) like everytime I parse Projector lamp hours from various projectors. And I have never seen the GetString/Copystring errors before. What bothers me is maybe I didn't check for them because I was getting the results I wanted. I went through the rest of the program and found the same error in the projector lamp parsing, and in 2 different devices check sum calculation.

    And as it ends up my touch panel lock up was not caused by the errors - but by a run away while loop.

    Any thought on this?
  • DHawthorneDHawthorne Junior Member Posts: 4,584
    I've seen a number of mysterious GetString errors, and they were generally similar to yours - the string passed to the function was not in a form the function was expecting. There are no line numbers in the debug output because it is a NetLinx library function, and you don't have the source for them, much less a debug-info compiled version or even the original source. The error is internal to the function. In those cases, it would be very helpful if the run time debugging was able to tell you what line of code called the function that generated the error.
  • JohnMichnrJohnMichnr Junior Member Posts: 276
    DHawthorne wrote:
    In those cases, it would be very helpful if the run time debugging was able to tell you what line of code called the function that generated the error.

    yes it would be... I ended up putting a bunch of send string 0 through out various areas to track down where in the code the errors popped up.
Sign In or Register to comment.