Receiving serial from real devices

Spannertech

This morning I was attempting to add the ability to get feedback from a Lutron Homeworks system. I know a module exists. I had all the outbound strings setup and working fine. I just needed to add parsing of LED feedback, so I naively *thought* it would be simpler just to do my own code

Well, I got it working, but only about 90% reliably. It seemed to come down to speed. If a scene was unique to a Lutron virtual keypad, then it was fine as I'd only get one feedback string for that particular panel. But, if the scene was available at several keypads, then it might or might not work, because I'd get a LED string (something like: "'KLS [xx.xx.xx] 0101010....',$0d") from every keypad that just changed its LED pattern in very quick succession. It seemed the AMX didn't catch and act on them all. Just to keep you on your toes, you also get a host of other responses you could do without but can't turn off.

It's all about what goes on here:

Data_event[dvDevice]
{
string:
{
the stuff in here
}
}

It seems receiving from virtual devices, it's always OK to just use data.text. But with real devices, I've always felt more confident using Create_buffer. I'm on the hunt for THE fastest possible way. I'm kind of curious what goes on inside AMX's comm modules. I seem to have the best success with setting a variable as the event is triggered, thus:

Data_event[dvDevice]
{
string:
{
if (!n)
{
n = 1
while find_string etc in here
}
n = 0
}
}

But this appears still to not be fast enough for getting these Lutron strings reliably. When I added some send_string 0 diagnostics, I saw that the event was triggered several times when I don't think it should have.


The other thing I have to do is run a loop on all those 1s and 0s and fill an array which I then equate to the buttons in a separate feedback timeline. This is the relatively easy part, once you've got the string properly isolated.

A bit of output in the "working case" where only 1 panel is sending back a feedback string:

>(0000238492) KBP, [1:4:20], 4 //debug showing outbound message (push)

(0000238591) KBR, [1:4:20], 4 //debug showing outbound message (release)

(0000238658) RAW STRING IS //unexpected trigger

(0000238659) RAW STRING IS //unexpected trigger

(0000238660) RAW STRING IS KBP, [01:04:20], 4//expected this one

(0000238661) RAW STRING IS//unexpected trigger

(0000238662) RAW STRING IS DL, [01:01:00:04:02], 100 //expected this one

(0000238784) RAW STRING IS //unexpected trigger

(0000238785) RAW STRING IS
//unexpected trigger

(0000238786) RAW STRING IS KLS, [01:04:20], 100100000000000000000000 //expected this one

(0000238787) STRING IS KLS, [01:04:20], 100100000000000000000000 //expected this one - the only part I actually want

(0000238788) PAN IS 20 //debug announcing the panel address

(0000238791) RAW STRING IS
//unexpected trigger

(0000238792) RAW STRING IS KBR, [01:04:20], 4//expected this one

Any thoughts gratefully received....

OP

mediapilotinc

Re: Receiving serial from real devices

Originally posted by Spannertech
Data_event[dvDevice]
{
string:
{
if (!n)
{
n = 1
while find_string etc in here
}
n = 0
}
}

Looks like one of the problems is that you could start processing before all data has arrived; The "n" variable is set and cleared independent of what has arrived.

Here is a sample of a routine I often use:

DATA_EVENT[dvDEV]
{
STRING:
{
DEV_DATA = " DEV_DATA,DATA.TEXT" // fill the heap
WHILE (FIND_STRING(DEV_DATA,"$0D",1)) // end of data section found; "$0D" would be the end of data indicator, please adjust as required
{
DEV_BUFFER = REMOVE_STRING(DEV_DATA,"$0D",1) // remove each section of data from heap
// process the DEV_BUFFER data section while new data can come in
}
}
}

This ensures that received all data gets processed (as long as buffer sizes are adequate, you can use the CREATE_BUFFER routine if necessary)

Chip Moody

Oliver, I've had AMX tell me on more than a few occasions that the fastest/most efficient way of getting into a buffer from a COM port is to still use the CREATE_BUFFER command in DEFINE_START, then use the DATA_EVENT as an alert to your program to check the contents of the buffer for anything useful. (And if not, dump out and wait for the next trigger)

Also, what processor and COM port are are you using for the Lutron?

- Chip

DHawthorne

I personally hate sifting through events to find what I need. I prefer to have the DATA_EVENT fire a CALL. But I have also found that relying on the STRING handler also sometimes loses data: if a lot of stuff comes in at once, you get one event. If you aren't careful to process all of it, you could find you missed some.

So my answer is to buffer it, and call my parsing routine with the STRING handler in the DATA_EVENT. At the end of my parsing CALL, I have it call itself if there is any more data in the buffer. You have to be a little careful with this, recursive function calls can eat up memory fast, but if you design it so they can't stack too deep they do the job nicely.

Parsing Lutron strings is easy because they are clearly delimited. But they can also be very chatty at times. You can reduce a great deal of your overhead by turing off dimmer level monitoring in the Lutron panel. I only leave keypad events and keypad LED strings active, and GRX feedback if there are any in the system.

As for efficiency in parsing LED strings, you can isolate the data by breaking outthe LED string itself with a simple REMOVE_STRING after you have your address determined, then assigning it wholesale to a string array. There is no need to do it element by element - your feedback can just look at the individual elements and do a character compare rather than numeric. For example:

CHAR sBuffer[1024]
CHAR sIncoming[100]

CHAR sCmd[3]
CHAR sAddress[10]
CHAR sLED[24]

sIncoming = REMOVE_STRING(sBuffer, "13", 1)
sCmd = LEFT_STRING(REMOVE_STRING(sIncoming, "', '", 1), 3)
sAddress= LEFT_STRING(REMOVE_STRING(sIncoming, "', '", 1), 10)
sLED = LEFT_STRING(sIncoming, 24) // strip final CR

----

// feedback

[dvTP, nLED1] = (sLED[1] == '1') // repeat for all LEDs

Of course, this is a bit simplified; I generally make all my KP information a structure that breaks out processor, link, and keypad number, as well as button text and LED state. Then I make an array of those structures with elements for every keypad in the job. The address processing determines where in the array to place the data as it comes in, and the feedback section iterates through all keypads currently requiring display.

mediapilotinc

Placing the processing in a CALL rather than under the DATA_EVENT is more a stylistic issue than a functional one, the processing gets done in the same sequence using either way. One could argue that using a CALL would slow things down slightly as the processor would have to jump to a different section of the processing stack and then jump back.

The WHILE loop in my example ensures (as long as the arrays are large enough) that all sections of the code are processed, one section at a time untill no more <end-of-data> characters are found (in this case the carriage return ($0D, 13) since it will repeat itself within the DATA_EVENT.

It should be easy to filter for relevant data within the WHILE loop. For Oliver's sample above I would suggest the following
DATA_EVENT[dvDEV]
{
STRING:
{
DEV_DATA = " DEV_DATA,DATA.TEXT" // either fill the heap here or use a CREATE_BUFFER in the DEFINE_START section per AMX's advice
WHILE (FIND_STRING(DEV_DATA,"$0D",1)) // end of data section found
{
DEV_BUFFER = REMOVE_STRING(DEV_DATA,"$0D",1) // remove each section of data from heap

IF (FIND_STRING(DEV_BUFFER,'KLS',1))
{
JUNK = REMOVE_STRING(DEV_BUFFER,'KLS, [',1) // toss now irrelevant part of data
// now process the keypad and LED info left in the string
// Dave showed a good way to process the data
}
}
}
}

If one is interested in data sections besides those containing 'KLS', a SELECT - ACTIVE routine instead of the IF statement would be an efficient method of processing. The key is to only look for stuff you're interested in and ignore the rest as fast as possible.

I am aware that the JUNK assignment is optional here and could be left out. Not sure if code runs faster with or without the JUNK assignment, I just like to know where stuff goes.

Spannertech

Dave, I guess I'm curious about your technique you describe in this paragraph:

"So my answer is to buffer it, and call my parsing routine with the STRING handler in the DATA_EVENT. At the end of my parsing CALL, I have it call itself if there is any more data in the buffer. You have to be a little careful with this, recursive function calls can eat up memory fast, but if you design it so they can't stack too deep they do the job nicely."

It sounds like you've done this and have it working. Can you elaborate a bit on this technique? Parsing the LED strings is easy enough and not the part I'm scared of. What I observe is the AMX missing stuff. You set up the string event to look for a "$0d", remove that and work on it, but it seems like another string too soon after will sometimes get missed.

Thanks!

Reese Jacobs

Receiving serial from real devices

Oliver,

One thing, having written a number of HomeWorks modules, the Lutron processor seems to return null responses on occasion (a string with a single byte and a CR or just a CR) and I have had to code specially in the module to ignore strings of length 1 or 2 since meaningful response is always longer than 1 or 2 bytes. This might help explain why the STRING handler fires sometimes when you do not expect it and when there is really nothing there for you to process. It is a little annoying since it wastes processor cycles for a non-event but it is easily handled nonetheless.

Reese

Chip Moody

Originally posted by DHawthorne
At the end of my parsing CALL, I have it call itself if there is any more data in the buffer. You have to be a little careful with this, recursive function calls can eat up memory fast, but if you design it so they can't stack too deep they do the job nicely.

Dave, If I'm reading correctly, that would be something like:

DEFINE_FUNCTION ParseBuf ()
{
  IF (FIND_STRING(MyBuf,"$0D",1))
  {
    ThisMsg = REMOVE_STRING (MyBuf,"$0D',1))
    // Parse contents of ThisMsg
    ParseBuf ()
  }
}

In what ways would that work differently from, or what Pros/Cons are there with this:

DEFINE_FUNCTION ParseBuf ()
{
  WHILE (FIND_STRING(MyBuf,"$0D",1))
  {
    ThisMsg = REMOVE_STRING (MyBuf,"$0D",1))
    // Parse contents of ThisMsg
  }
}

I would think both would accomplish the same results without any problems, with the latter not invoking recursion - no?

I have also recently started putting code like this in function calls rather than leaving it in the DATA_EVENT -> STRING handler myself. Instead of tracking down a particular DATA_EVENT for the code I'm looking for, I can just hit the "Goto Sub" button on my toolbar and pick "ParseXYZBuf" from the drop-down list...

Thanks,
- Chip

DHawthorne

Yes, Chip illustrated what I was talking about accurately. This was not somethig you could do in Axcess, but is used commonly in C++, which is where I got started in the programming world.

And yes, the WHILE loop he showed would do the exact same thing in this example, and probably would be a better choice for parsing a buffer.

I'm not certain at all why I settled on recursive calls, but it has become habit. Most likely it's just the first hting I thought of to make sure a buffer gets fully processed, and I just never looked further. It's not a danger unless your data flow is such that it recurses faster than the data gets processed - if that happens, you quickly eat up all available memory, and the entire system locks solid . Now you could get the same thing if you get stuck in a WHILE loop, but it's a bit harder to do.

Now I'm sure there are situations when it would be better to use a recursive call, but I can't think of one offhand. That's one of the great things about a forum like this - different perspectives on things you never looked at twice can expand your repertoire and make you a better developer.

Chip Moody

Thinking about it, I would think it would take a REALLY chatty device to make either method screw up the system...

- Chip