Serial Data Parsing Question on Projector video wall
JohnMichnr
Posts: 279
Hey All - I'm looking for help on a video wall that I am programming. I have 24 cube wall that I want to parse status on during power on & power off. I have a timeline sending out the status command to each wall for the duration of the poweron/off cycle ( I shut the status command off for the cubes that are either full on or full off) Then once a wall reports full on I ask for lamp 1 hours, the responce to that triggers the request for lamp 2 hours. At first everything seemed like it was working.... (like starting a novel with It was a dark and stormey night...)
But - I my program started locking up and the system was throwing out diagnostics error messages GetString - Error1 and CopyString Error-1. As I was looking into that I noticed that the wall would sometimes stop responding for 4-5 seconds - then send out a tons of responces, and some garbage thrown in there just for fun. (It responded with a power responce for a cube that was not there)
Now the problem that I am having is that the responce data has a start byte, no end byte, and is differing lengths (either 7 or 9 bytes long) I have to make sure that I have full and correct responce byte before hitting my parsing. And the start byte is $02 that can be used anywhere in the command - not just the start byte.
Here is a typical responce: $02,$04,$A0,$01,$00,$00,CS,$01
First $02 - Start Byte
$04 - data length
$A0 - Command (this case status)
$01,$00,$00 - three bytes of data (plust the command above equals the $04 for data length)
CS - check sum two's complement of length command and data (previous post)
$01 - what projector is responding.
Here is what I am using:
This code work fine to parse teh data, but only when I have a full and complete string going into it.
I need some help getting the data into the while loop. I think I need to create a loop that grabs each byte one at a time (get_buffer_char) then if the byte is a $02 - look 2 byte ahead and if it is a $A0 or $A1 (the only 2 I am working on) go back one byte to the data length and get that many bytes plus 2 (check sum and which projector)
But I am not that sure how to do it. Any suggestions? Any other thoughts?
Thanks in advance...
But - I my program started locking up and the system was throwing out diagnostics error messages GetString - Error1 and CopyString Error-1. As I was looking into that I noticed that the wall would sometimes stop responding for 4-5 seconds - then send out a tons of responces, and some garbage thrown in there just for fun. (It responded with a power responce for a cube that was not there)
Now the problem that I am having is that the responce data has a start byte, no end byte, and is differing lengths (either 7 or 9 bytes long) I have to make sure that I have full and correct responce byte before hitting my parsing. And the start byte is $02 that can be used anywhere in the command - not just the start byte.
Here is a typical responce: $02,$04,$A0,$01,$00,$00,CS,$01
First $02 - Start Byte
$04 - data length
$A0 - Command (this case status)
$01,$00,$00 - three bytes of data (plust the command above equals the $04 for data length)
CS - check sum two's complement of length command and data (previous post)
$01 - what projector is responding.
Here is what I am using:
string:
{
stack_var char sMsg[9]
stack_var integer nWhich
stack_var char nLamp
stack_var char nTempLamp[3]
stack_var integer nSize
stack_var char nCmd
stack_var integer N
stack_var integer nLocation
sWallRxbuf = "sWallRxbuf,data.text"
while(find_string(sWallRxbuf,"$02,$04,$A0",1) || find_string (sWallRxbuf,"$02,$05,$A1",1)
|| (LENGTH_STRING(sWallRxbuf) > 8))//lets parse this
{
sTrash = remove_string(sWallRxbuf,"$02",1)
nSize = get_buffer_char(sWallRxbuf)+2
if(length_string(sWallRxbuf)<nSize)
{
sWallRxbuf = "$02,nSize-2,sWallRxbuf" //put back what you just took out
}
else
{
sMsg = GET_BUFFER_STRING(sWallRxbuf,nSize)
nCmd = GET_BUFFER_CHAR(sMsg)
switch (nCmd)
{
case $A0: //status command
{
cBit[1] = sMsg[1] & 1
cbit[2] = sMsg[1] & 2
cbit[3] = sMsg[1] & 4
cbit[4] = sMsg[1] & 8
cbit[5] = sMsg[1] & 16
cbit[6] = sMsg[1] & 32
cbit[7] = sMsg[2] & 1
cbit[8] = sMsg[2] & 2
cbit[9] = sMsg[2] & 4
cbit[10] = sMsg[2] & 8
cbit[11] = sMsg[2] & 16
cbit[12] = sMsg[2] & 32
cbit[13] = sMsg[2] & 64
cbit[14] = sMsg[3] & 1
cbit[15] = sMsg[3] & 2
cbit[16] = sMsg[3] & 16
cbit[17] = sMsg[3] & 32
cbit[18] = sMsg[3] & 128
sTrash = get_buffer_string(sMsg,4)
nWhich = get_buffer_char(sMsg)
//cCubeRespCounter[nWhich]=0
for(N=1;N<=6;n++)
{
if(cbit[n]>0)
cCubeStatus[nWhich] = N-1
}
(*if(cLastCubeStatus[nWhich]<>cCubeStatus[nWhich])
{
call 'CUBE STATUS' (nWhich)
cLastCubeStatus[nWhich]=cCubeStatus[nWhich]
}*)
switch (cCubeStatus[nWhich])
{
case 4: //ERROR ON CUBE
{
cCubeError[nWhich] = ''
for(N=7;n<=18;n++)
{
cCubeError[nWhich] = "cCubeError[nWhich],';',nCubeErrorTxt[n]"
}
}
case 0: //cube is off
{
off[cCubeCheck[nWhich]]
off[cCooling[nWhich]]
}
case 5: //cube is on
{
off[cWarming[nWhich]]
off[cCubeCheck[nWhich]]
call 'WALL XMIT' (nGetLamp1,nWhich)
}
default:
{
on[cCubeCheck[nWhich]]
}
}
}
case $A1: //lamp hours status
{
nLamp = get_buffer_char(sMsg)
nTempLamp = get_buffer_string(sMsg,3)
sTRASH = get_buffer_char(sMsg)
nWhich = GET_BUFFER_CHAR(sMsg)
cCubeRespCounter[nWhich]=0
switch (nLamp)
{
case $20: //lamp 1
{
cCubeLampHours[nWhich][1] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]
call 'WALL XMIT' (nGetLamp2,nWhich)
}
case $40: //Lamp 2
{
cCubeLampHours[nWhich][2] = 65535*nTempLamp[1] + 256*nTempLamp[2] + nTempLamp[3]
}
}
}
}
}
}
}
This code work fine to parse teh data, but only when I have a full and complete string going into it.
I need some help getting the data into the while loop. I think I need to create a loop that grabs each byte one at a time (get_buffer_char) then if the byte is a $02 - look 2 byte ahead and if it is a $A0 or $A1 (the only 2 I am working on) go back one byte to the data length and get that many bytes plus 2 (check sum and which projector)
But I am not that sure how to do it. Any suggestions? Any other thoughts?
Thanks in advance...
0
Comments
If $02 is present
and next (data length) byte is present
{
Throw away everything before the $02
if at least the correct amount of data ($02 + 1 + data length + 2) is present
{
Cut the string out of the incoming stream leaving the rest for later
if the checksum is correct (call a separate subroutine)
and the projector number is sensible
{
Parse and interpret the comand
}
}
}
I buffer the entire response, and peel off char by char. Generally, right after the start char is some indication of how big the packet is going to be: either a response code that has a specific data length to follow, a byte telling you how much data is to follow, or a combination of those. With that, you can build up your response in a holding variable, and finish it up when you have all the data you are expecting. If you get a new start character before the packet finished, throw it away (and perhaps send your debug window a message in case there is something wrong and not just junk data).
My experience is that equipment that uses this kind of protocol tends to be working at a fairly low level on the serial port; it's not bufered or interpreted within the device, and therefore more prone to errors on the output. So you have to be more careful with your error handling and make sure your code will deal with it gracefully rather than barfing all over your design.
Your doParseVidWall function needs to have a local variable of the proper size for a single packet. What it needs to do is take the highest order character off the buffer, and append it to this local variable. For triggering purposes, I would also make a single char for holding each chunk so you can test it and decide what to do. You could do something like:
DEFINE_FUNCTION doParseVidWall() { STACK_VAR CHAR cDigit[1] LOCAL_VAR CHAR sPacket[256] cDigit = LEFT_STRING(sVidWallBuffer, 1) // get first character sVidWallBuffer = RIGHT_STRING(sVidWallBuffer, LENGTH_STRING(sVidWallBuffer) - 1) // remove character from buffer IF(cDigit == "$02) CLEAR_BUFFER sPacket // we have a start char, clear data and start over ELSE { sPacket = "sPacket, cDigit" // append new data to the packet variable IF(LENGTH_STRING(sPacket) == <amount of data we are expecting> <code to process packet> } }I'm sure there are other ways, that's just how I've handled similar situations. It would seem to me though, that the bulk of your problem is the STRING handler. What determines when that will fire on a NetLinx is when the master sees a pause in the data stream, it assumes the data is finished. So you are very likely to get incomplete data bursts using it if anything at all, including normal serial port delays in the equipment, makes that data pause. It will be in the next burst, it's not lost, but that is likely to throw off your parsing. Buffering it and testing char by char eliminates this. In my opinion, the STRING handler on a port is only suitable when you know for certain you will always get a full and complete packet of data every time - which is almost never with a serial port. I personally always buffer serial ports, and only use that handler with intra-master virtual messages.
This is the approach I?ve used in the past. It?s similar to what NMarkRoberts eluded to earlier. Here is some code:
DEFINE_DEVICE dvProj = 5001:1:0 DEFINE_CONSTANT INTEGER nMinLength = 7 INTEGER nCmdPad = 3 //the $02 + checksum byte,+ proj# byte DEFINE_VARIABLE CHAR cBuffer[1024] DEFINE_FUNCTION fnParseBuffer () { CHAR junk[64] INTEGER junksize INTEGER cmdsize CHAR cmd[64] //do we at least have the minimum requirements for a valid message? IF (LENGTH_ARRAY(cBuffer)>=nMinLength && FIND_STRING(cBuffer,"$02",1)) { //lets check to see if any garbage is in fromt junksize = FIND_STRING(cBuffer,"$02",1) - 1 IF (junksize) { //strip the garbage out junk = GET_BUFFER_STRING(cBuffer,junksize) SEND_STRING 0, "'**** JUNK - ',ITOA(junksize),'junk characters getting tossed - ',junk" //lets spin out if the junk reducted our length to less than minimum IF (LENGTH_ARRAY(cBuffer) < nMinLength) RETURN } //we're still here so lets find out how long this command is cmdsize = cBuffer[2] + nCmdPad IF (LENGTH_ARRAY(cBuffer)>=cmdsize) { //do we have the entire command here? //we do so lets strip it cmd = GET_BUFFER_STRING(cBuffer,cmdsize) IF (fnCheckChecksum(cmd)) { //if checksum is valid fnProcessCommand(cmd) //pass this command on to be processed. } ELSE { SEND_STRING 0, "'Invalid checksum received with cmd: ',cmd" } } ELSE { SEND_STRING 0, "'Don''t have entire response yet'" //time to spin out and try again next go around RETURN } } } DEFINE_FUNCTION INTEGER fnCheckChecksum (char incoming[]) { //do whatever to check the checksum RETURN 1 //if checksum is good } DEFINE_FUNCTION fnProcessCommand (char cmd[]) { //see if it's a command you want to do something with and do it } DEFINE_START CREATE_BUFFER dvProj,cBuffer DEFINE_EVENT DEFINE_PROGRAM //I usually have this in mainline. //It's probably more correct to put it in a TIMELINE //Even more correcter (huh?) I think would be to put //this in the STRING: handler for dvProj but feel I //might miss something if I do. I'm curious where others put it fnParseBuffer() (***************THE END****************)HTH
But in answer to your question in the commented out portions: I would put it in mainline too. For all the reasons mentioned above, I don't think the STRING section is quite reliable for every possible situation where it might get only partial data. Perhaps it's just paranoia.
Again Thanks!
Joe I'll name a comemorative sub routine for you in my program...
Let us know how the story ends.
I knew you were going to say that...
Actually I spent 6 quality hours with a 4 cube version on the wall yesterday with mixed results. The code that Joe sent out (thanks again) worked mostly, a few mode here and there and it seemed to come together. The big problem is that the wall goes stupid 5 seconds in and then waits 5 seconds and spits out a ton of stuff, some of it good, some of it not so good. The other thing the wall does is put out an ACK responce to some commands, any command going to cube #2, of $10,$02 like the same $02 that is supposed to be a start code for the responces... So the buffer gets one character off because it has 2 $02 in a row. But I can remove that just by looking for that ACK responce and deleting it before it gets to the buffer
The real issue now is that the Hebert Code (TM) and my parsing code is throwing out a bunch of GetString error 1 and CopyString Error 1 responces in diagnositcs. I don't know if that is cuaseing the other big error...
The customer did not want to purchase a touchpanel so we are using a G3 web panel for the control. That panel keeps locking up when I fire up the video wall as a whole. I can fire up individual cubes using another page and not have it lock up, but firing up the wall as a whole locks up the control panel, Sometimes. The only difference on the whole wall firing is that I add 2 more timelines going for a countdown timer on the webpanel and a general timer for the startup sequence of the system. So I don't know if that is the problem, or what. I don't have anymore time infront of the wall until I go onsite next week, but I programmed another cahssis to be a video wall emulator (except for the 5 second stupid period - but I am considering adding that) so I will be testing with my emulator until then.
The funny thing with this is - Ive been working so hard to get teh status to come up on a touchpanel page - I never really wondered if I really needed it. The wall consistan'y turns on & off with teh commands, I have never had that fail. Just problems with the status parse. So I am thinking of backing down my requirement a little.
There shouldn?t be any cases where the code I posted (unmodified) will cause any runtime errors.
Been there done that.
By the way - the debug info did not show any line numbers for these issues - don't know why
From Joes original code above ( And I did it as well a bunch of times)
DEFINE_CONSTANT nCmdPad = 3 DEFINE_VARIABLE CHAR cBuffer[1024] DEFINE_FUNCTION fnParseBuffer () { CHAR junk[64] INTEGER junksize INTEGER cmdsize CHAR cmd[64] cmdsize = cBuffer[2] + nCmdPadcmdsize is an integer, and 16 bits, 1 byte of cBuffer is a char and 8 bits. Relating an 8 bit byte to the 16 bit integer would through up those errors - as below from the Diagnostic window.
tech support wanted me to BAND the single 8 bit byte with $FF before equating it to the integer. as below going through and removing all of those in Joe's code and my own code got rid of all the errors.
The part I don't get is that the first code worked - even though it through out the error, and I know that I have done that routine before (equating a single portion of a character array to an integer variable) like everytime I parse Projector lamp hours from various projectors. And I have never seen the GetString/Copystring errors before. What bothers me is maybe I didn't check for them because I was getting the results I wanted. I went through the rest of the program and found the same error in the projector lamp parsing, and in 2 different devices check sum calculation.
And as it ends up my touch panel lock up was not caused by the errors - but by a run away while loop.
Any thought on this?
yes it would be... I ended up putting a bunch of send string 0 through out various areas to track down where in the code the errors popped up.