Network outages and Master to Master
travis
Posts: 180
I have the pleasure of maintaining a 7 master system on a Cisco Meraki network that is administered by a different contractor. Apparently there are scheduled, automatic, not-cancellable firmware updates that happen to the switches, routers, and firewalls.
When this happens, the AMX system breaks. Some of the masters won't even ping until they are rebooted. It seems like others just aren't able to re-get the M2M going again until they are rebooted.
Anyone seen anything like this?
I'm using the Cascading topology for the master to master. Route mode direct.
When this happens, the AMX system breaks. Some of the masters won't even ping until they are rebooted. It seems like others just aren't able to re-get the M2M going again until they are rebooted.
Anyone seen anything like this?
I'm using the Cascading topology for the master to master. Route mode direct.
0
Comments
What do you mean by cascading topology? Typically if a master loses contact with a master in its URL it will keep trying indefinitely, and then reconnect once its back on the network. This sounds like a network issue more than an AMX master issue. especially if you can't ping a master.
Paul
This is a kludgy and inelegant fix, but it will force the network data to be refreshed. I had at least one case where if I rebooted (ie., power cycled) a dumb network switch that was intermediate, that also restored my connection. Try to find the minimum amount of things to kick, then kick them when you need to.
It allows you to auto ping addresses, to reboot the various outlets, if a subsequent number of pings fail.
It may not be the most ideal solution, but it's a $129 option.
I've seen AMX masters and touch panels drop their IP address when our network engineers push out config changes to our Cisco switches. During the update process the Cisco gear drops all layers except hardware - i.e. the link-layer disappears, but the electrical connection is maintained. (Using OSI terminology, the physical layer remains but the data link layer disappears.) This confuses the AMX gear in a random* manner, where some units end up with an "address" of 0.0.0.0. Unplugging the network cable for at least 5 seconds and replugging it resolves the issue. (Removal for 3 seconds is not long enough!) Getting the network engineers to temporarily shut down the network port ("shut" then "no shut" in Cisco-speak) for 10 seconds also works. I try to get them to do this when they are making major updates to many rooms/buildings.
*Whilst I keep all my firmware versions the same (per model) and try to minimize variations in AMX hardware models, the distribution of the issue is still random across my fleet. The NDX-CV7 panels seem to be more susceptible than other AMX devices.
I doubt that M2M has any bearing on the issue you are having. It is probably a lower network-related issue and how each piece of AMX hardware copes with the network outages.
Expanding on Dave H's thought by a miniscule amount: If you want to avoid having code to auto-reboot your master, maybe you could put a small unmanaged Ethernet switch inline to each master and cycle the power on the switch when required. One switch per master. This would break/make your network connection when required.... Of course, this assumes that 1) you have a spare relay and 2) a disconnect/reconnect of the network lead resolves the issue.
Roger McLean
Swinburne University.
I wonder if the new nx masters have NICs like G5 panels...
You could telnet into 127.0.0.1 and then try to ping something.
Or monitor your URL list and try to decide based on how many M2M connections are missing.
I've been browsing through the netlinx help and none of the built-in commands are jumping out at me.
GET_IP_ADDRESS always returns an address in my tests.
A way to catch M2M errors like
Line 2741 (14:40:38.422):: Closing connection to 10.10.0.33 due to duplicate (0:1:3 already exists)
in the program would be nice too.
Here's the current version but with out the part that get's the public ip commented out. The code posted in the modpedia section should have the missing constants, the ones that are global to all my files, if not let me know. I have another function in my code, another axi that does all my timing, dates and stuff like that that may be in the posted code but basically in there there's a function that calls every minute routines that triggers this code by calling the check minute function. You can also use send commands to call it form where ever to want and it can return a response. It should be easy to figure out.