Home AMX User Forum AMXForums Archive Threads AMX Hardware

COM port issues

I ran into this recently... here is an email that I sent to Those Who Care within my company. Hope it saves some of you spending an afternoon banging your head against the wall.

--- cut here --- 8< ---

Be aware that AMX's NI-series controllers have some limitations with respect to their built-in RS232 ports. Due to a very small (64-byte) receive buffer on these ports, they will not work properly with devices that produce an even moderately verbose response to commands.

Today I encountered this on-site: an NI-4000's built-in COM port would control the Tandberg 2500... mostly. But when I tried to retrieve speed-dial directory listings from the codec, they would not parse properly even though I've been using the identical code for this purpose for a couple of years now. It turns out that it was because the Tandberg directory responses are in the neighborhood of 68 to 80 bytes, depending on the phone number and name.

The built-in COM ports are OK for Tandberg use if you're NOT doing anything fancy. "Fancy" means really anything beyond basic control: things like retrieving directory entries, doing far-end control of a non-VISCA camera, getting the far end's phone number...

Other "chatty" gear will run into the same problem: certain Sony equipment responds with 80-byte packets, and Pesa routing switcher status responses are about 145 bytes for a 16-output switch.

This is only an issue with the NI* built-in ports. NXIs and NXC-COM2 cards have a 196-byte buffer, so they're good for most applications. For gear that's extremely yakkity, we may want to investigate TCP/IP control if it's available.

The bottom line: watch out what control equipment you spec, and which device gets connected where. Yes, it means we need to know up front a) fairly detailed present and future user requirements and b) a good bit about the control protocol for the equipment under consideration.

Comments

  • Spire_JeffSpire_Jeff Posts: 1,917
    I ran into this problem a while ago with a B&K CT610. I never took the initiative to narrow it down to a 64 byte buffer, but in retrospect, that's exactly the problem I was having. It is a really easy thing to fix. All you have to do is create a buffer in your program and then parse the buffer for a complete command when the data event gets triggered. If anyone needs a quick example of code, let me know and I'll dig it out.

    Jeff
  • The thing I have to ask right off the bat - where did someone >tell< you that a 64-byte hardware buffer was a bad thing, and that you couldn't use chatty devices with them?

    I'm not sure who's code you're using/writing, but in reading your message, one would think that I would NOT be able to use this block of code:
    DEFINE_DEVICE
    
    Comm = 5001:2:0
    		  
    DEFINE_VARIABLE
    
    VOLATILE CHAR    CommBuf[10000]
    
    DEFINE_START
    
    CREATE_BUFFER Comm,CommBuf
    
    DEFINE_EVENT
    
    DATA_EVENT [Comm]
    { ONLINE:
      {
        SEND_COMMAND Comm,'SET BAUD 57600,N,8,1 485 DISABLE'
        SEND_COMMAND Comm,'HSOFF'
      }
      STRING:
      {
        SEND_STRING 0,"'Buffer size = ',ITOA(LENGTH_STRING(CommBuf))"
        WAIT 20 CommBuf = ''
      }
    }
    

    and successfully capture messages larger than 64 bytes, right?

    In that case, when I ran the above code on an NI2000 and set up another controller to blast 4000 bytes out one of it's com ports, I should not have been able to get this when I telnetted into the NI2000 and turned messages on:
    (0000462778) Buffer size = 64
    (0000462789) Buffer size = 128
    (0000462800) Buffer size = 192
    (0000462811) Buffer size = 256
    (0000462823) Buffer size = 320
    (0000462834) Buffer size = 384
    (0000462845) Buffer size = 448
    (0000462856) Buffer size = 512
    (0000462867) Buffer size = 576
    (0000462878) Buffer size = 640
    (0000462889) Buffer size = 704
    (0000462900) Buffer size = 768
    (0000462912) Buffer size = 832
    (0000462923) Buffer size = 896
    (0000462934) Buffer size = 960
    (0000462945) Buffer size = 1024
    (0000462956) Buffer size = 1088
    (0000462967) Buffer size = 1152
    (0000462978) Buffer size = 1216
    (0000462989) Buffer size = 1280
    (0000463000) Buffer size = 1344
    (0000463014) Buffer size = 1408
    (0000463025) Buffer size = 1472
    (0000463034) Buffer size = 1536
    (0000463045) Buffer size = 1600
    (0000463056) Buffer size = 1664
    (0000463067) Buffer size = 1728
    (0000463078) Buffer size = 1792
    (0000463089) Buffer size = 1856
    (0000463100) Buffer size = 1920
    (0000463112) Buffer size = 1984
    (0000463123) Buffer size = 2048
    (0000463134) Buffer size = 2112
    (0000463145) Buffer size = 2176
    (0000463156) Buffer size = 2240
    (0000463167) Buffer size = 2304
    (0000463178) Buffer size = 2368
    (0000463189) Buffer size = 2432
    (0000463200) Buffer size = 2496
    (0000463212) Buffer size = 2560
    (0000463223) Buffer size = 2624
    (0000463234) Buffer size = 2688
    (0000463245) Buffer size = 2752
    (0000463256) Buffer size = 2816
    (0000463267) Buffer size = 2880
    (0000463278) Buffer size = 2944
    (0000463289) Buffer size = 3008
    (0000463300) Buffer size = 3072
    (0000463311) Buffer size = 3136
    (0000463323) Buffer size = 3200
    (0000463334) Buffer size = 3264
    (0000463345) Buffer size = 3328
    (0000463356) Buffer size = 3392
    (0000463367) Buffer size = 3456
    (0000463378) Buffer size = 3520
    (0000463389) Buffer size = 3584
    (0000463400) Buffer size = 3648
    (0000463411) Buffer size = 3712
    (0000463423) Buffer size = 3776
    (0000463434) Buffer size = 3840
    (0000463445) Buffer size = 3904
    (0000463456) Buffer size = 3968
    (0000463486) Buffer size = 4000
    

    I mean, I'm trying not to sound flame-y, but I am having a hard time with that. It >sounds< like you had a problem with losing characters in your code and >didn't< seek help from tech support. I would think that if you had, and someone at AMX said "Oh yeah, you're just expecting too much from our hardware - it can't hack that! You're SOL on an NI with that!", you would have said as much. Instead, it sounds like you had a problem, didn't seek assistance with it, and essentially went and unfoundedly told "those who care" that NI com ports suck, and shouldn't be used with high-traffic devices.

    I'm not sure if it's been officially documented, but I know there have been discussions here with regards to >certian< NI series com ports that are "less robust" than others - notice I used com 2 with the sample code... Likewise, at 115k I lost bytes, but I also can't say whether the problem was with the controller that was sending the data or not. Then again, how many A/V devices do you know of that are going to send 4k chunks of data at a time at 115k?

    Hey, otherwise, if I'm out of line on this folks, please feel free to flame back... There was just something about the original message that came across as wrong - maybe it's just me.

    - Chip
  • chillchill Posts: 186
    Originally posted by Chip Moody
    The thing I have to ask right off the bat - where did someone >tell< you that a 64-byte hardware buffer was a bad thing, and that you couldn't use chatty devices with them?

    Nobody told me it was a bad thing. It's an empirical finding, and I thought I'd help out my fellow programmers by sharing. Sorry if that annoys you.

    I'm not sure who's code you're using/writing, but in reading your message, one would think that I would NOT be able to use this block of code:

    {snipped some code to keep the size down]
    CREATE_BUFFER Comm,CommBuf
    
        SEND_STRING 0,"'Buffer size = ',ITOA(LENGTH_STRING(CommBuf))"
        WAIT 20 CommBuf = ''
    

    and successfully capture messages larger than 64 bytes, right?
    [snip]
    In that case, when I ran the above code on an NI2000 and set up another controller to blast 4000 bytes out one of it's com ports, I should not have been able to get this when I telnetted into the NI2000 and turned messages on:
    (0000462778) Buffer size = 64
    (0000462789) Buffer size = 128
    (0000462800) Buffer size = 192
    (0000462811) Buffer size = 256
    
    [some output snipped]
    Irrelevant. I'm not talking about a "buffer" as in "create_buffer" - I'm talking about the RAM on the COM port that holds the data short-term before it appears in your char array. (In hardware engineering terminology this is called a buffer.) If the incoming packet size is larger than that RAM, you will lose data.

    I mean, I'm trying not to sound flame-y, but I am having a hard time with that. It >sounds< like you had a problem with losing characters in your code and >didn't< seek help from tech support. I would think that if you had, and someone at AMX said "Oh yeah, you're just expecting too much from our hardware - it can't hack that! You're SOL on an NI with that!", you would have said as much. Instead, it sounds like you had a problem, didn't seek assistance with it, and essentially went and unfoundedly told "those who care" that NI com ports suck, and shouldn't be used with high-traffic devices.

    Actually, I did seek assistance. Maybe it wasn't clear in my original post, but this code has been running flawlessly on about 40 NXIs and card frames for more than two years. When I ran it on an NI4000, I started losing characters. When I used a COM2 card in one of the 4000's slots instead of the built-in port, the problem went away.

    I called AMX tech support, and Pete Stauber told me about this hardware buffer issue and said that others had had similar problems. I've spoken with Pete in the past, and I consider him to be one of the best, most knowledgeable people I've ever dealt with at AMX. I'm confident that he knows what he's talking about. In light of all this, I don't think my statements were unfounded.

    I'm not sure if it's been officially documented, but I know there have been discussions here with regards to >certian< NI series com ports that are "less robust" than others

    It could be that a different port would have worked OK, but I was not in a position to experiment.

    - notice I used com 2 with the sample code... Likewise, at 115k I lost bytes, but I also can't say whether the problem was with the controller that was sending the data or not. Then again, how many A/V devices do you know of that are going to send 4k chunks of data at a time at 115k?

    None. But a Tandberg 2500 *does* spew 70 or 80 bytes at 9600, and that's enough. BTW, baud rate doesn't matter here; it's the RAM buffer size that's the issue.

    Hey, otherwise, if I'm out of line on this folks, please feel free to flame back... There was just something about the original message that came across as wrong - maybe it's just me.

    - Chip
  • Spire_JeffSpire_Jeff Posts: 1,917
    The code posted by Chip is what I was refering to. As Chip elegantly put it, it works just fine. I have been using the above system with no problems for a while now.

    On a seperate note:
    *Passes Chip a Jolt Cola and a new (non smashed against the floor) XBOX controller and urges him to breathe in the good air, and out with the bad. ;) *

    Jeff
  • Spire_JeffSpire_Jeff Posts: 1,917
    Originally posted by chill
    Irrelevant. I'm not talking about a "buffer" as in "create_buffer" - I'm talking about the RAM on the COM port that holds the data short-term before it appears in your char array. (In hardware engineering terminology this is called a buffer.) If the incoming packet size is larger than that RAM, you will lose data.

    I'm not sure if this was made clear to you, but as far as I have experienced, using the CREATE_BUFFER line does fix this problem. I experience no data loss from the B&K CT610 which can generate strings easily in excess of 115 bytes. Implementing the CREATE_BUFFER line allows the NI-3000 to handle things just fine. It just creates a string event every 64 bytes. When this string event happens, I just check the buffer for the START_OF_COMMAND sequence and the END_OF_COMMAND Sequence and if both are found, I run my handler on the buffer.

    I am not trying to flame anyone here, just trying to provide my experiences (and opinions) in an effort to help others.

    Jeff
  • Originally posted by chill
    Nobody told me it was a bad thing. It's an empirical finding, and I thought I'd help out my fellow programmers by sharing. Sorry if that annoys you.

    Only annoys me when the information isn't correct or is misleading. :) In those cases, it's not considered helping...

    Irrelevant. I'm not talking about a "buffer" as in "create_buffer" - I'm talking about the RAM on the COM port that holds the data short-term before it appears in your char array. (In hardware engineering terminology this is called a buffer.) If the incoming packet size is larger than that RAM, you will lose data.

    Since your NetLinx code doesn't give a squat about how large the physical buffer is, it is >quite< relevant. I believe the information I posted previously shows quite well that data coming in that's larger than the hardware buffer size can >still< get moved into "your char array" without getting corrupted.


    I called AMX tech support, and Pete Stauber told me about this hardware buffer issue and said that others had had similar problems. I've spoken with Pete in the past, and I consider him to be one of the best, most knowledgeable people I've ever dealt with at AMX.

    Pete is definately one of the best guys down in TX, no doubt. While I wasn't present for the conversations you had with him, I doubt very much that he would have told you that you that >any< AMX processor - even an Axcent3 - would have difficulty with 80-byte packets...


    ...a Tandberg 2500 *does* spew 70 or 80 bytes at 9600, and that's enough.

    Did you miss in my example where FOUR THOUSAND bytes were spewed in an instant? At 56k? Obviously 70 or 80 bytes at 9600 baud is trivial.


    BTW, baud rate doesn't matter here;...

    Since the baud rate determines how quickly your buffer - hardware or otherwise - fills up, um... yeah, it kinda does.


    ...it's the RAM buffer size that's the issue.

    The point that you seem to have missed in my posting is that the size of the hardware (RAM) buffer is not an issue in a system that's running fast enough to get the data out of said hardware buffer and into your software before getting overwritten by new data. There simply isn't any reason to be concerned about the size of the physical buffer when you're not supposed to be parsing or otherwise manipulating data that's in there in the first place. C'mon, they cover that in the level 1 programming courses for cripes sake.

    - Chip
  • Originally posted by Spire_Jeff
    *Passes Chip a Jolt Cola and a new (non smashed against the floor) XBOX controller and urges him to breathe in the good air, and out with the bad. ;) *

    Jeff


    Thanks Jeff - more caffeine than I normally get with my Pepsi - woohoo! :)

    Now, where do I get a replacement for that shattered Halo 2 disc?...

    (And for the record, I ranked up last night, so today's attitude is an exception)
  • chillchill Posts: 186
    Originally posted by Chip Moody
    Only annoys me when the information isn't correct or is misleading. :) In those cases, it's not considered helping...

    I don't believe it's incorrect. If it was misleading I can only apologize, blaming my mediocre language skills for my failure to communicate clearly.

    Since your NetLinx code doesn't give a squat about how large the physical buffer is, it is >quite< relevant. I believe the information I posted previously shows quite well that data coming in that's larger than the hardware buffer size can >still< get moved into "your char array" without getting corrupted.

    Of course you can receive data all day long with no problems; I do it all the time. But I'm not sure your test was an accurate replication of the situation I had. (See below.) BTW, I said "your char array" to avoid using the B-word; that word seemed to be confusing things.

    Pete is definately one of the best guys down in TX, no doubt. While I wasn't present for the conversations you had with him, I doubt very much that he would have told you that you that >any< AMX processor - even an Axcent3 - would have difficulty with 80-byte packets...

    Your doubt does not alter the fact that he said it. And just to clarify, when I say "packet" I mean a non-stop burst of data, like a datagram.

    Did you miss in my example where FOUR THOUSAND bytes were spewed in an instant? At 56k? Obviously 70 or 80 bytes at 9600 baud is trivial.

    I didn't miss it, I'm just not convinced it means anything in this context. BTW, how long is "an instant"? How much time did it actually take, and what was delay between characters? How did you measure these things?

    Since the baud rate determines how quickly your buffer - hardware or otherwise - fills up, um... yeah, it kinda does [matter].

    Baud rate is not the same thing as character pacing. Think about it.

    The point that you seem to have missed in my posting is that the size of the hardware (RAM) buffer is not an issue in a system that's running fast enough to get the data out of said hardware buffer and into your software before getting overwritten by new data.

    I didn't miss your point; I just think you're mistaken :^) To reiterate my own point, it sure LOOKS like an issue from here. Acts like one, too.

    Here is some speculation, based on what Pete was telling me: it seems that there needs to be some sort of pause in the data stream - however brief - so the system can read the data out of the COM port buffer and into the memory space where your software buffer is. I could be way off base with this; corrections are welcome, flames are not.

    There simply isn't any reason to be concerned about the size of the physical buffer when you're not supposed to be parsing or otherwise manipulating data that's in there in the first place.

    So what's your explanation? Consider:
    - the same code worked fine in many NXIs and NXFs (I've programmed dozens of VTC suites for this client);
    - this problem of lost data has occurred on both NI-series we've put in for this client (one NI4000 and one 3000), never on an NXI or NXF;
    - Pete tells me this is a known problem with the NI-series;
    - the problem went away when I used a COM card instead of the 4000's internal COM port, with NO CODE CHANGES other than the device declaration.

    Frankly, it didn't for a moment occur to me that all of the following were true: a) my code is broken even though it worked in the first 45 systems; b) that business with the NI4000 and NI3000 was pure coincidence; c) Pete was either mistaken or lying to me; d) the card working where the internal port didn't was also coincidental. In fact, I don't believe any of them are true.

    Seriously, what do you think was going on? How do you explain this? If the lost data problem was *not* due to a too-small RAM buffer, what was the cause?

    C'mon, they cover that in the level 1 programming courses for cripes sake.

    That was uncalled-for and unprofessional. C'mon, I'm trying to stay civil here; a little help?

    I've been programming since the Nixon administration, and there is a thing I've learned which I'd like to pass along: Don't be so sure you know everything about the equipment you use. Familiarity != knowledge. AMX has done a fine job of making their gear easy to program and use, but there's a LOT going on under the hood. I know enough about it to at least grasp the breadth of my ignorance, but you may wish to reconsider your certainty regarding a situation you have apparently never encountered.

    Have a nice weekend.
  • Explanation

    You guys are both right from what I am reading here. What the difference is the technique to parse data.

    Apparently the NIxxxx has a smaller Com port hardware buffer than a NXC-COM2 card or an NXI Com port.

    If you are parsing data in a Data_Event and only using the Data.Text object, it is possible to lose characters in an NIxxxx program if the data packet exceeds the hardware buffer. This would represent the total content (Length_String) of Data.Text during the Data_Event.

    On the other hand, if you parse data from a character array that is referenced to a Create_Buffer statement you will not lose data. This because the processor will continue to append the character array up to its defined size. So instead of parsing data from the Data.Text object use the character array.

    BTW, this second approach was the way it was taught in Axcess before NetLinx was released.

    Chill, I understand your frustration because you experienced unexpected results from your program in an NIxxxx. Perhaps this is one of the reasons why the NIxxxx is less expensive than a NXI.
  • Originally posted by chill
    BTW, how long is "an instant"? How much time did it actually take, and what was delay between characters? How did you measure these things?

    Since the only other controller I had handy at the time was a Cres tron Pro2, I threw together a quick Cres tron program that contains one of their "String I/O" symbols. (This was the only logic symbol in the program, FWIW) I created 16 fields on this symbol, each one containing 250 characters. I took a button press from the controller's front panel to trigger all 16 strings at once, so I'd say that the 4000 bytes flew out the port as quickly as it could possibly send them. In this case I define an "instant" as the period of time indicated by the short blip of the com port's TX led on the Pro2.

    If you like, I'll recreate the same thing on another NetLinx processor as soon as I can - I imagine I'd use a chunk of code like this:
    BUTTON_EVENT [TP,1]
    { PUSH:
      {
        LOCAL_VAR INTEGER x
    
        FOR (x=1;x<=16;x++) SEND_STRING RS232Port,'<insert 250 characters here>'
      }
    }
    

    This would have to be run on something other than an NI series, since the NI >output< buffers have or had a limitation on 64 characters...

    So what's your explanation? ... If the lost data problem was *not* due to a too-small RAM buffer, what was the cause?

    I can only speculate with the info available at the moment, not explain. :) At this point, I would have to ask if you were parsing your RS232 data without the use of a buffer, by accessing DATA.TEXT directly in a DATA_EVENT's STRING: handler. If that's the case, that would be the answer right there. The other controllers / com cards you've used have larger hardware buffers that would allow you to get away with that with the devices you're trying to control.

    If you are using a buffer, I would speculate that you've connected the serial line to one of the NI series COM ports that's known to have issues... (The ones reported here in the forums by other users, not the 64k thing)



    you may wish to reconsider your certainty regarding a situation you have apparently never encountered.

    The only certainty I have is with the tests I've done, and experience I've had with equipment I've worked with in the past - which includes more string parsing than I care to think of. Everything else in this thread is combination of speculation and skepticism. :)

    - Chip
Sign In or Register to comment.