parsing long string with no delimiter

postich · June 2012

i am self taught programmer so my methods will not have that formal look or feel. Also my programming skill is limited so please, if you comment, be easy on me.

OK, so i want to get the run hours for a NEC projector's lamp, filter panel ect..
The proj responds with a 104 bit HEX string with an known beginning but no delimiter.
The string comes in broken up into 2 data events.
The code i came up with only works in a perfect world where the only data coming in is that 104 bit string.

$23,$8A,$1,$10,$62 is the known beginning and the last 17 bit is the data i want.

    string:
    {
	local_var char cProjectorString[256]
	cProjectorString="cProjectorString,data.text"
	if((find_string(cProjectorString,"$23,$8A,$1,$10,$62",1))&&(length_string(cProjectorString)=104))
	{
	    cProjectorInfo=right_string(cProjectorString,17)
	    cProjectorString="''"
	}
    }

I know the code above is wrong and i could use some guidance on handling this kind of problem.

ericmedley · June 2012

I tend to not rely on using the data event to process the string. The only reliable string you can count on being in one piece at data event fire is internal stuff like messages from a virtual device from a module or stuff like that.

I tend to make a buffer, building it myself from he data event or even use the old Create Buffer for really large messages like web page scrapes. The idea is that you create a function to check and see if the buffer contains all the stuff you need and then process the string only then. You use the data event to fire the checking routine.

I then fire the parsing routine from a timeline that is only on when there's stuff waiting in the buffer. An empty buffer kills the timeline. But there's many other ways to do this.

Once I'm sure I have a complete message, I then send it to the parsing routine.

mush · June 2012

postich wrote: »

$23,$8A,$1,$10,$62 is the known beginning and the last 17 bit is the data i want.

G'day Postich,

I too, am completely self taught.
Your code may have bugs or perhaps could have been written better but it's never "wrong".
What will help you here is the knowledge that the 5th byte of the return string from NEC projectors is the length of the string - 6 bytes. eg The length of the string in your example is 104 bytes long. 104 - 6 = 98 ($62) which is what the 5th byte is!
So, some code..

STRING:
    {
	local_var char cProjectorString[256]
	cProjectorString="cProjectorString,data.text"
        If (Length_String(cProjectorString) = (6 + cProjectorString[5]))  // If we have a complete string
	{
	    cProjectorInfo=right_string(cProjectorString,17)
	    cProjectorString="''"
	}
    }

I hope this helps.
Good luck!

Mush

postich · June 2012

oh yeah!

oh yeah it is, that helps. when ever i read api guides i tend to just skim over the pages until i see what i need. must have missed that piece of valuable info.

vining · June 2012

You should also verify there's no garbage in the beginning of the buffer "cProjectorString". Of course that depends on how many variation there may be for the leading bytes prior to the return length bytes [5].

If the known beginning is all that gets sent or all you'll ever care about receiving.

$23,$8A,$1,$10,$62 is the known beginning and the last 17 bit is the data i want.

Then I would add additional code to verify my string begins with this with no junk prior.

STRING:
     {
     local_var char cProjectorString[256]
     stack_var integer nFBS;
     
     cProjectorString="cProjectorString,data.text"
     nFBS = find_string(cProjectorString,"$23,$8A,$1,$10",1);//less the length byte in case this is common to all types of returns
     if(nFBS > 1)
	  get_buffer_string(nFBS-1);//remove all beginning crap
     if(nFBS)
	  {
	  SELECT
	       {
	       ACTIVE{Length_String(cProjectorString) = (6 + HEXTOI(cProjectorString[5]))):  // If we have a complete string
		    {
		    SEND_STRING 0, "'NEC PROJ RX, MATCH FOUND, STR-[ ',cProjectorString,' ]'";
		    cProjectorInfo=right_string(cProjectorString,17)
		    cProjectorString="''"
		    }
	       ACTIVE(1):
		    SEND_STRING 0, "'NEC PROJ RX, NO MATCH FOUND!, STR-[ ',cProjectorString,' ]'";
	       }
	   }
     }

vining · June 2012

Also you should add an else as I did below so the buffer doesn't fill with crap that isn't needed which could pontentially cause you to receive the first 4 bytes you're looking for and drop some of the following bytes. If I recall if this type of buffer is full you won't be able to append more data into it like a created buffer can so you have to prevent it from filling with any unwanted junk in the first place.

I could be wrong but I think a created buffer will keep shifting to the left as more data comes in from the right and these non created buffers don't, once they fill they won't accept any more data. Could someone else confirm this is actually correct.

STRING:
     {
     local_var char cProjectorString[256]
     stack_var integer nFBS;
     
     cProjectorString="cProjectorString,data.text"
     nFBS = find_string(cProjectorString,"$23,$8A,$1,$10",1);//less the length byte in case this is common to all types of returns
     if(nFBS)
	  {
	  if(nFBS > 1)//moved so it doesn't run twice unless string is found
	       get_buffer_string(nFBS-1);
	  SELECT
	       {
	       ACTIVE{Length_String(cProjectorString) = (6 + HEXTOI(cProjectorString[5]))):  // If we have a complete string
		    {
		    SEND_STRING 0, "'NEC PROJ RX, MATCH FOUND, STR-[ ',cProjectorString,' ]'";
		    cProjectorInfo=right_string(cProjectorString,17)
		    cProjectorString="''"
		    }
	       ACTIVE(1):
		    SEND_STRING 0, "'NEC PROJ RX, NO MATCH FOUND!, STR-[ ',cProjectorString,' ]'";
	       }
	  }
     else
	  cProjectorString="''";
     }

postich · June 2012

Thanks!

I used different parts of the code that you guys showed and came up with this final (for now)string handler.

    STRING:
    {
	local_var char cProjectorString[2048]
	
	cProjectorString="cProjectorString,data.text"
	if(Length_String(cProjectorString)=(6+cProjectorString[5]))  	//complete string found
	{
	    fnParseProj(cProjectorString)
	    cancel_wait 'hold'
	    cProjectorString="''"
	}
	else
	{
	    wait 1 'hold'
	    {
		cProjectorString="''"
	    }
	}
    }

the nec will have different beginnings for string depending on the command sent so i didn't use find_string in the event, instead i look only for a complete string and if i find one i send it to parse.

i used a wait to allow time for incomplete string to arrive, not sure if that is best but it works.(10th of a second sould be enough you think?)
as far as error checking and garbage handling, well... i think that if the 5th byte = the total length then it is a good bet i have valid info and if not the parser should be able to ignore it with find_string.
what i like about this is that this code can be use with any similar protocol.

its funny how simple the solution was but i could not see it until i studied the code you guys posted, super cool, thanks.

P.S. HEXTOI was not applicable in my instance, still have yet to find a use for it.

vining · June 2012

If there's any junk in the beginning of your string your 5th bit won't be the intended length bit and you'll not likely achieved a length string match on that bit what ever value that might be in the 5th bit position. It has the potential of missing incoming strings because of this. Any junk at all in the beginning will wreck the process.

Are you goiing to process the other possible returns? If not don't worry about them but if you are list the various beginnings that you'll want to process so we can suggest a more reliable option.

I assume since there are different beginnings the differences indicate the type of return and the format of the data that follows which would be the key to proper string parsing.

postich · June 2012

here is my code for displaying an NEC projector, lamp1 lamp2 and filter in hours.

iv tested it for a few hours, bombarding it with all kings of commands, everything seems OK so far. have not seen any strange behavior yet.

here is the parser fn

DEFINE_VARIABLE
persistent long nProjectorHrs[5]

DEFINE_FUNCTION fnParseProj(char stringdata[])
{
    local_var char debug[2048]
    local_var char cTemp[16]
    debug=stringdata
    if(find_string(stringdata,"$23,$8A,$1,$10,$62",1))
    {
	cTemp=mid_string(stringdata,88,16)
	nProjectorHrs[4]=((cTemp[8]*16777216)+(cTemp[7]*65536)+(cTemp[6]*256)+(cTemp[5]))/3600
	send_command dvTP,"'^TXT-8,0,',itoa(nProjectorHrs[4]),' hrs.'"		// projector total hrs
	nProjectorHrs[1]=((cTemp[16]*16777216)+(cTemp[15]*65536)+(cTemp[14]*256)+(cTemp[13]))/3600
	send_command dvTP,"'^TXT-5,0,',itoa(nProjectorHrs[1]),' hrs.'"		// filter hrs
	send_string dvDISPLAY,"cProjLamp1"					
    }
    if(find_string(stringdata,"$23,$9B,$01,$10,$07,$00,$00,$00",1))
    {
	cTemp=mid_string(stringdata,9,4)
	nProjectorHrs[2]=((cTemp[4]*16777216)+(cTemp[3]*65536)+(cTemp[2]*256)+(cTemp[1]))/3600
	send_command dvTP,"'^TXT-6,0,',itoa(nProjectorHrs[2]),' hrs.'"		// lamp1 hrs
	send_string dvDISPLAY,"cProjLamp2"
    }
    if(find_string(stringdata,"$23,$9B,$01,$10,$07,$01,$00,$00",1))
    {
	cTemp=mid_string(stringdata,9,4)
	nProjectorHrs[3]=((cTemp[4]*16777216)+(cTemp[3]*65536)+(cTemp[2]*256)+(cTemp[1]))/3600
	send_command dvTP,"'^TXT-7,0,',itoa(nProjectorHrs[3]),' hrs.'"		//lamp2 hrs
    }
}

here is the datd event

    STRING:
    {
	local_var char cProjectorString[2048]
	
	cProjectorString="cProjectorString,data.text"
	if(Length_String(cProjectorString)=(6+cProjectorString[5]))  	//complete string found
	{
	    fnParseProj(cProjectorString)
	    cancel_wait 'hold'
	    cProjectorString="''"
	}
	else
	{
	    wait 1 'hold'
	    {
		cProjectorString="''"
	    }
	}
    }

postich · June 2012

here is my code for displaying an NEC projector, lamp1 lamp2 and filter in hours.

iv tested it for a few hours, bombarding it with all kings of commands, everything seems OK so far. have not seen any strange behavior yet.

here is the parser fn

DEFINE_VARIABLE
persistent long nProjectorHrs[5]

DEFINE_FUNCTION fnParseProj(char stringdata[])
{
    local_var char debug[2048]
    local_var char cTemp[16]
    debug=stringdata
    if(find_string(stringdata,"$23,$8A,$1,$10,$62",1))
    {
	cTemp=mid_string(stringdata,88,16)
	nProjectorHrs[4]=((cTemp[8]*16777216)+(cTemp[7]*65536)+(cTemp[6]*256)+(cTemp[5]))/3600
	send_command dvTP,"'^TXT-8,0,',itoa(nProjectorHrs[4]),' hrs.'"		// projector total hrs
	nProjectorHrs[1]=((cTemp[16]*16777216)+(cTemp[15]*65536)+(cTemp[14]*256)+(cTemp[13]))/3600
	send_command dvTP,"'^TXT-5,0,',itoa(nProjectorHrs[1]),' hrs.'"		// filter hrs
	send_string dvDISPLAY,"cProjLamp1"					
    }
    if(find_string(stringdata,"$23,$9B,$01,$10,$07,$00,$00,$00",1))
    {
	cTemp=mid_string(stringdata,9,4)
	nProjectorHrs[2]=((cTemp[4]*16777216)+(cTemp[3]*65536)+(cTemp[2]*256)+(cTemp[1]))/3600
	send_command dvTP,"'^TXT-6,0,',itoa(nProjectorHrs[2]),' hrs.'"		// lamp1 hrs
	send_string dvDISPLAY,"cProjLamp2"
    }
    if(find_string(stringdata,"$23,$9B,$01,$10,$07,$01,$00,$00",1))
    {
	cTemp=mid_string(stringdata,9,4)
	nProjectorHrs[3]=((cTemp[4]*16777216)+(cTemp[3]*65536)+(cTemp[2]*256)+(cTemp[1]))/3600
	send_command dvTP,"'^TXT-7,0,',itoa(nProjectorHrs[3]),' hrs.'"		//lamp2 hrs
    }
}

here is the data_event

    STRING:
    {
	local_var char cProjectorString[2048]
	
	cProjectorString="cProjectorString,data.text"
	if(Length_String(cProjectorString)=(6+cProjectorString[5]))  	//complete string found
	{
	    fnParseProj(cProjectorString)
	    cancel_wait 'hold'
	    cProjectorString="''"
	}
	else
	{
	    wait 1 'hold'
	    {
		cProjectorString="''"
	    }
	}
    }

parsing long string with no delimiter

Comments