Home AMX User Forum NetLinx Studio

AMX closing IP port? How long to wait on IP connection?

I?ve been playing w/wireshark >.< I have my PC, an extron device, and the controller connected to a dumb hub w/static IP addresses on the same subnet.

I?m watching communication w/extron devices. I know typically by default Extron has a 300 second (5minute) timeout on the IP connection.

I sent this command: SEND_STRING data.device,"$1B,'1*65000TC',$0D"

This sets the global extron timeout to 650,000 seconds (each increment = 10 seconds) ? that?s over 7 days.

SO?..

I open a connection on port 23. I send some data. After *60* seconds it looks like AMX is doing something to check the connection every 10 seconds.

111 65.530566 192.168.4.200 192.168.4.60 TCP [TCP Keep-Alive] cap > telnet [ACK] Seq=4 Ack=109 Win=17520 Len=0
124 75.555652 192.168.4.200 192.168.4.60 TCP [TCP Keep-Alive] cap > telnet [ACK] Seq=4 Ack=109 Win=17520 Len=0
137 85.580823 192.168.4.200 192.168.4.60 TCP [TCP Keep-Alive] cap > telnet [ACK] Seq=4 Ack=109 Win=17520 Len=0
150 95.605845 192.168.4.200 192.168.4.60 TCP cap > telnet [RST, ACK] Seq=5 Ack=109 Win=17520 Len=0

Then I get the message in diagnostics:

Exiting TCP read thread ? closing this socket for the local port 11.
cIPEvent::Offline 0:11:1

My connection definitely closes.

I decided to see how long port 23 will stay open if I just telnet into the extron device. I open a telnet session and wait, wait, wait?
It doesn?t seem to close (I?m not going to wait 7 days to verify ? it is definitely longer than 5 minutes).

So it seems like AMX is closing the port. Is this true? Is it new?

Last time I programmed an Extron switcher via IP was 2 years ago and I know it didn?t do this. I was able to hold a connection open for days.

It isn?t necessarily a problem. I wrote code to open a connection every time I need to send something, and queued up all the commands that are waiting for the connection to actually open. I?d rather the port stayed open longer. Maybe that is taboo for networking?

I did notice a nice thing about the IP connection though ? there is something really wants to ensure my strings get sent. If I open a connection, pull the plug, then send commands, there is a retransmission that occurs at 1s, 2s,4s,8s,16s,32s. Finally, something throws in the towel and the connection closes.


As an aside how do other programmers typically handle that little bit of lag between when you TELL the connection to open and it actually opening and the device being ready?
Naturally now that I?m trying this again, I haven?t gotten the errors and everything is behaving. Here is what I noticed though.

When I did this:

Open Client
Wait for client to open
Send String

Sometimes the connection opened, the device said ?hello, I?m here?, wait wait wait wait wait wait, some errors in diagnostics/spam in wireshark, strings sent

I was able to get rid of the errors/spam/waiting by doing this:

Open client
Wait till client open && device says ?hello, I?m here?
Wait a little more
Send String

Comments

  • DHawthorneDHawthorne Posts: 4,584
    I've controlled many devices where I was perfectly able to leave the IP port open on the AMX perpetually. This is the very foundation of my device logging module. It always has a telnet port open for as long as the master is running. If anything, I would suspect the Extron is closing the port, not the AMX. In such cases, the device closes the port after a full exchange (typical of HTML, annd I have seen like behavior in other types of devices). I don't think the timeout applies, because the intent of that is it's still waiting, it will give up after a time ... if it feels the exchange is finished, it's not a timeout, it's just done and closes.
  • I'm guessing the problem is that AMX doesn't support TCP keepalive, while the command-line telnet application does. Not 100% sure on that. The fix is simple -- send something every 45 seconds or so to keep the connection open.

    I don't generally worry about opening and closing TCP connections... just open them and leave them. If you need to worry about it for some reason, put commands in a buffer, then start sending from the buffer once you receive the "connected" response from the device.
  • KariKari Posts: 13
    what is different?

    I'm thinking jweather might be right, but I am not network savvy...

    I can open up almost indefinite telnet sessions from my PC - Extron is not closing port 23.

    Yes, with AMX I can hold the port open OR just open it when I need it and send buffered commands (this is what I'm doing)

    What I'm wondering is - what changed? Or what could be affecting this differently than before?

    I didn't used to have to ping the port OR continually reopen it.
  • a_riot42a_riot42 Posts: 1,624
    Is there some reason you need to keep the connection open? Why not just reopen it as needed?
    Paul
  • HedbergHedberg Posts: 671
    a_riot42 wrote: »
    Is there some reason you need to keep the connection open? Why not just reopen it as needed?
    Paul

    Extron switchers will send unsolicited status change messages. If you close the connection, won't you miss these?
  • a_riot42a_riot42 Posts: 1,624
    Hedberg wrote: »
    Extron switchers will send unsolicited status change messages. If you close the connection, won't you miss these?

    The port would still be listening for a passive open. If a port needed to already be open in order to communicate, you couldn't telnet to any server without advance knowledge of the imminent connection.
    Paul
  • HedbergHedberg Posts: 671
    a_riot42 wrote: »
    The port would still be listening for a passive open. If a port needed to already be open in order to communicate, you couldn't telnet to any server without advance knowledge of the imminent connection.
    Paul


    Well, yes, you need to have a connection in order to communicate. The Extron device has a built-in server -- I think of it as a telnet server, though I don't actually know if it technically meets the definition of a telnet server or not. In order for the Extron server to send information to a client, the client needs to connect to the server and that connection needs to be maintained and active. The server will not connect to a disconnected client and send information, that's not how servers work. Also, on the AMX side, there is a client which is set up to contact the Extron server, This client will not accept a connection -- it's not a server, it's a client. If you want the AMX to accept (not initiate) a connection from some device, there needs to be a server, not a client.

    Some Extron devices may have telnet clients which will contact and connect to a telnet server and transfer information, but that's not the sort of connection we're discussing. Unless, of course, I've entirely missed the boat in this thread.
  • viningvining Posts: 4,368
    Hedberg wrote:
    Unless, of course, I've entirely missed the boat
    I don't suppose we should go there!

    This confuses me too, if it works like a telnet port once the port is opened it should maintain the connection. So who's closing it? The Extron telnet server shouldn't be unless it just works unlike other telnet servers and AMX shouldn't be closing it unless there's a problem with the connection or it's being told to close via code. Is this a Duet mod? Is this a known to work correctly mod? Do you have another Extron device to test?
  • KariKari Posts: 13
    devices tested

    I've tested w/the DVS304 and the IPL T SFI244 Extron devices - both are behaving identically.

    I'm not using a module. I'm just opening the port in Netlinx, then sending strings. I don't have anything that tells a port to close. Both devices stay infinitely open in telnet w/my PC. Both devices drop after ~60 seconds w/AMX. This happens after 3 StayAlive events I see in Wireshark.

    DEFINE_FUNCTION Open_Client(INTEGER ip_index)
    {
    //if port is not open and not being opened, open it
    IF(!_ipdev[ip_index].isconnected && !_ipdev[ip_index].inprocess)
    {
    IP_CLIENT_OPEN(_ipdev[ip_index].localport,_ipdev[ip_index].serveraddr,_ipdev[ip_index].serverport,_ipdev[ip_index].protocol)
    _ipdev[ip_index].ACK=0
    _ipdev[ip_index].inprocess=1
    }
    }

    I don't *need* the port to stay open. This is more of a...hrm...I'm not timing out in Telnet, why am I timing out in AMX?




    **edit**
    I just tried this with the same results....

    DEFINE_DEVICE
    dvtp=10011:1:1
    dvextron=0:5:0


    DEFINE_EVENT
    BUTTON_EVENT[dvtp,1]
    {
    PUSH:
    {
    ip_client_open(5,'192.168.4.72',23,IP_TCP)
    }
    }
  • HedbergHedberg Posts: 671
    Kari wrote: »
    I've tested w/the DVS304 and the IPL T SFI244 Extron devices - both are behaving identically.

    I'm not using a module. I'm just opening the port in Netlinx, then sending strings. I don't have anything that tells a port to close. Both devices stay infinitely open in telnet w/my PC. Both devices drop after ~60 seconds w/AMX. This happens after 3 StayAlive events I see in Wireshark.


    Ok. Apparently the Netlinx TCP client implements an optional TCP feature called "keep alive". This doesn't actually keep the connection active, what it does is close the connection when a proper response is not received from the server. The Netlinx client appears to be hardwired to have this feature turned on. It seems to be set to send a probe after 60 seconds of inactivity and then, if no response is received from the server, to send two more probes at 10 second intervals and then close the connection if no response is received within 10 seconds of the last probe.

    I checked this with the telnet server (using wireshark) that comes with XP Pro and the Netlinx TCP client sends this probe every 60 seconds and the XP server responds immediately. When connecting to an Extron IPL-T-S2, however, no response to the probe is generated by the Extron server and the Netlinx TCP client closes after 90 seconds.

    So, the behavior you are seeing is a combination of the Netlinx client wanting a response and the Extron server not providing one.

    I last experimented with the IPL-T-S2 a couple years ago and my recollection is that the Netlinx client did not shut down like it does now, but I can't recall for sure.

    According to TCP documentation that I've looked at, a proper implementation of the "keep alive" feature should allow it to be turned on and off and should allow for setting the times and the number of repeat probes. If the Netlinx TCP client allows for this control, I can't see how -- there doesn't appear to be an "IP_SET_OPTION" applicable.
  • a_riot42a_riot42 Posts: 1,624
    Could it have anything to do with this note in the manual?

    "The processor can send out unsolicited information (such as notice of a volume or input change or a change in some other setting). That is called verbose (wordy) relationship
    between the processor and a connected device. For a direct RS-232/422 connection, the processor is set for Verbose mode by default. When the DVS 304 is connected via
    Ethernet, Verbose mode is disabled by default in order to reduce the amount of communication traffic on the network. If you want to use the Verbose mode with a processor
    connected via Ethernet, this mode must be set to On each time you reconnect to the processor.."
    Paul
  • The NetLinx TCP/IP sockets have always set the KEEPALIVE option automatically upon socket creation. This is to ensure a timely detection of a dead socket in order to trigger an OFFLINE event back to the NetLinx application. I have no explanation as to why the Extron device is no longer responding to the TCP/IP keepalive query. Possibly a new version of Extron firmware?

    The new IP_SET_OPTION API was added to NetLinx recently to allow manipulation of particular socket options. To date the only options supported are IP_MULTICAST_TTL_OPTION and IP_TCP_NODELAY_OPTION. The TTL (time-to-live) option allows a multicast packet to be transmitted beyond the local subnet. The NODELAY option forces all outgoing TCP sends to be transmitted immediately rather than being buffered to ensure that all data is transmitted prior to a socket being closed.
  • NetLinx TCP/IP sockets have always had the KEEPALIVE option automatically set at socket creation. This is to ensure a timely detection of a dead socket in order to trigger an OFFLINE event back to the NetLinx application. I have no explanation as to why the Extron modules are no longer responding to the keepalive queries. Possibly a newer version of Extron firmware?

    The IP_SET_OPTION API was added recently to provide the ability to modify particular socket optional parameters. To date on two options are supported, IP_MULTICAST_TTL_OPTION and IP_TCP_NODELAY_OPTION. The TTL option is to allow a multicast socket to be transmitted beyond the local subnet and the NODELAY option is to ensure that all outbound socket communication is transmitted prior to the socket being closed.
  • ColzieColzie Posts: 470
    Is the consensus that the Master is doing what it should, but the device being controlled isn't responding as expected to the behind the scenes "keep alive" queries?

    I have the same issue with Lutron. I want to stay connected to catch any unsolicited data, but the connection keeps closing. I have written code to reconnect every few seconds. While this works, it has the unfortunate side effect of filling the log with online/offline events (not to mention all the unnecessary communication). The log is a great way to see how the program is running over the period of a few days - providing the opportunity to catch any programming errors in a live system.

    It doesn't do much good, however, when every 7 hours the log is completely full of online/offline chatter:
    NI Log wrote:
    1 03-11-2010 THU 08:01:54 Interpreter CIpEvent::OnLine 0:5:1
    2 03-11-2010 THU 08:01:54 SocketManager Connected Successfully
    3 03-11-2010 THU 08:01:44 Interpreter CIpEvent::OffLine 0:5:1
    4 03-11-2010 THU 08:01:44 Interpreter Exiting TCP Read thread - closing this socket for local port 5
    5 03-11-2010 THU 08:00:14 Interpreter CIpEvent::OnLine 0:5:1
    6 03-11-2010 THU 08:00:14 SocketManager Connected Successfully
    7 03-11-2010 THU 08:00:04 Interpreter CIpEvent::OffLine 0:5:1
    8 03-11-2010 THU 08:00:04 Interpreter Exiting TCP Read thread - closing this socket for local port 5

    ...

    994 03-11-2010 THU 01:15:15 Interpreter CIpEvent::OffLine 0:5:1
    995 03-11-2010 THU 01:15:15 Interpreter Exiting TCP Read thread - closing this socket for local port 5
    996 03-11-2010 THU 01:13:45 Interpreter CIpEvent::OnLine 0:5:1
    997 03-11-2010 THU 01:13:45 SocketManager Connected Successfully
    998 03-11-2010 THU 01:13:35 Interpreter CIpEvent::OffLine 0:5:1
    999 03-11-2010 THU 01:13:35 Interpreter Exiting TCP Read thread - closing this socket for local port 5
    1000 03-11-2010 THU 01:12:05 Interpreter CIpEvent::OnLine 0:5:1

    Any thoughts?
  • jweatherjweather Posts: 320
    Send a harmless command every 45 seconds, like KLMON. This is the only reliable way to hold a TCP connection open. Otherwise, there are multiple timeouts at the device, the master, and possibly even intervening firewalls that can kill the connection.
  • kbeattyAMXkbeattyAMX Posts: 358
    Just ping the extron with your own keepAlive message that guarantees a response. If the extron does not respond, close the port and reopen it. You could run this off of the Offline Event but the extron may be really offline for greater than 90 seconds.
  • KurtKurt Posts: 16
    Extron IP connection via DUET module

    Would it be safe to assume the AMX DUET module for Extron switchers would take care of all this IP connection keep alive / reconnect problems without us having to do anything?

    Kurt Elfers - ACE
  • viningvining Posts: 4,368
    Kurt wrote: »
    Would it be safe to assume the AMX DUET module for Extron switchers would take care of all this IP connection keep alive / reconnect problems without us having to do anything?

    Kurt Elfers - ACE
    No, never assume anything. Don't even assume it will work at all. Try it and evaluate how it performs and see if anything needs to be done additionally. With luck you won't need to do anything else.
  • PhreaKPhreaK Posts: 966
    vining wrote: »
    No, never assume anything. Don't even assume it will work at all. Try it and evaluate how it performs and see if anything needs to be done additionally. With luck you won't need to do anything else.
    \
    Ha! True, sad but true. Excellent advice.
  • a_riot42a_riot42 Posts: 1,624
    Colzie wrote: »
    Any thoughts?

    You have an online, offline and error event for the socket so you always know whether its open or closed. If you want to stay connected to a socket on another device, the algorithm should be pretty simple. If your socket is open send your data with no worries. If your socket closes, open it so that asynchronous data can be received. I don't think its really any more complicated than that. In network code, you can't assume anything, so I don't bother with keep alive or pinging or any other workaround. Since I know whether the socket is connected or not at all times, I open it if I need it open and close it when I need to, and any errors that are generated I send to the console. This way you assume nothing, and it will always work if it can, otherwise an error is thrown. Are you looking for something beyond this functionality?
    Paul
  • ColzieColzie Posts: 470
    a_riot42 wrote: »
    Are you looking for something beyond this functionality?
    Paul

    Thanks for the reply, I had forgotten about this thread (it was over a year ago when I posted the question).

    If you'll reread my post you'll see I have everything working functionally (you explained exactly what I've been doing), my issue was the log file getting filled with all of the online/offline events.
Sign In or Register to comment.