Home› AMX User Forum› AMX Technical Discussion
Options

Recommended Max Duet Modules Running On An AMX Master

DigiMeDigiMe Posts: 59
in AMX Technical Discussion
I know this is a bit of a loaded question with a lot of unknown variables like:
1. Which AMX Master: NI-700, NI-2100, NI-3100 w/256, etc....
2. What Duet Modules: Tandberg, Extron, etc....
3. Any other code running?

So, let us just use an example of:
NI-2100 running only Tandberg Duet Modules with some code to gather data from several connected CODECs as well as basic control.

PS - Sure, I could just a write my own Non-Duet Tandberg module using telnet for the method of control, but lets say time is an issue and the Tandberg Duet Module is already developed and working.

Comments

  • Options
    catalinocatalino Posts: 25
    This really depends on a lot of things. What you are really concerned about is available amount of free memory specifically for duet modules(there's a telnet shell command that shows you how much free duet memory you can allocate) and how busy your code is. If you are constantly polling and parsing, that's going to affect your CPU usage. So far I'm at 2 duet modules and 7 netlinx modules on an ME-260/64 as my busiest system. I haven't stress-tested the equipment to see how much crap I can communicate with at once, but in theory you can use master-master communication to lighten the load on the main processor.
  • Options
    ericmedleyericmedley Posts: 4,177
    Yeah, there isn't going to be a real easy answer to this. The honest best and quickest way to find out is to load it up, get it running and look at the memory useage. That's going to be the fastest, most efficient way to find out a working real number.


    Some issues that make it really difficult to know.

    is this a case where you're going to have to put in multiple instances of the module to deal with multiple panel control/feedback or do you have a UI for it that deals with multiple panels? Are there multiple CODECs being controlled by multiple panels? etc....
  • Options
    DigiMeDigiMe Posts: 59
    There is not going to be a UI, per say. I am currently developing a C# program that would interface with the AMX master using the PCLink.ocx/dll as the pipe between the two.

    I know I could start loading up Tandberg Duet Comm modules and see the memory usage and performance. I just just seeing if I could find a rough number so I would know how many AMX masters I would need.

    PS - I know I could just write all the code in C# and not use any AMX masters. It is just I have been programming AMX for about 8 years. but C# for about 2 months or so. I actually have written a DLL file for controlling a Tandberg CODEC which works fine. I just did not want to rely on a "Windows" based box for everything. :)
  • Options
    HARMAN_ChrisHARMAN_Chris Posts: 598
    The type of device being controlled plays into the answer of your question. More sophisticated devices like video conference units and DSPs tend to have a great deal more going on under the hood to keep track of all of the controllable parameters. Generally speaking, one or two modules is not an issue. When going above that, you may find improved performance by increasing your duet memory allocation using the SET DUET command in telnet. The default is 8mb, but you could take it to 16mb without much compromise under most normal programming situations. I have cases where we have even exceeded 16mb, but I would not do it without cause or reason as there could potentially be other unintended consequences to your overall program based on your other defined elements (memory total is fixed, so adding to one area will subtract from another).

    When running multiple instances of the same duet module, please ensure that your firmware is at least 3.6.454 or higher because an issue was discovered and resolved pertaining to multiple instances of some modules.

    Several Helpful Telnet Commands for your master:
    cpu usage (reports the current processor load of your master)
    show mem (reports the current memory usage on your master)
    get duet (reports the current memory allocation for duet applications)
    set duet (sets the memory allocation for duet applications)
    show buffers (see if anything is hung in your java que)

    Unfortunately, you probably will not arrive at an answer to your question until you code the job and connect to physical devices. That will be the true test with polling and parsing. Not ideal, but probably the only true measure of performance given the high number of variables that go into determining system performance outside of the duet modules themselves.
  • Options
    John NagyJohn Nagy Posts: 1,734
    Chris, could you explain how to use CPU USAGE in telnet?
    It appears to always return a value it measured during boot. That is, no matter when I type CPU USAGE, I get exactly the same result...

    >cpu usage
    CPU usage = 99.99% (30 sec. average = 90.68%, 30 sec. max = 99.99%)

    This is no matter if the system is idle or processing intensely.

    I believe we've been told it has to be preceded with a REBOOT HEAP WATCH command, but others say this is superstitious and meaningless.
    It appears to do something:
    >reboot heap watch
    (0328457937) ConfigMgr::SetHeapWatchOn -Setting heap watch on and saving config
    (0328458112) Memory Available = 8469528 <10392>
    Rebooting in heap watch mode...

    However, after the reboot, CPU USAGE continues to give precisely the same result (to the 100th of a percent) anytime, idle or busy.
    Running firmware v3.60.447 on the unit I'm looking at now, but this has been the case anytime we've tried.

    Can you share the real answer?
  • Options
    HARMAN_ChrisHARMAN_Chris Posts: 598
    John,
    I can not speak to the number reported with the command and what the key drivers of this number are, but I see it spike at boot while things are loading and then it returns to a lower level after a few minutes.

    For instance, right now, the 3100 at my desk is running 7 duet modules at (duet mem at 16mb)-no gear hooked up:
    CPU usage = 14.94% (30 sec. average = 14.55%, 30 sec. max = 17.93%)

    a residential system running 9 duet modules at (duet mem at 10mb) - all devices connected and running:
    CPU usage = 15.63% (30 sec. average = 18.69%, 30 sec. max = 44.36%)

    The same residential project was running at 99% until I discovered two modules were using the same virtual device. When I resolved that, it dropped to the 15%. No other code change needed.

    Examples like this are why I say that there is a lot happening outside of the duet module that plays into big picture on how the system will run. I have never used the 'reboot heap watch' command. Just Friday I was testing a system and checked on the cpu usage and I too got the 99%, but just seconds later I looked again and it had dropped to around 20% and it stayed there once the boot processes were finished.

    Ultimately, the system will run at 99%. In many cases, that is sufficient. However, if you are experiencing lag, cpu usage and buffers can tell more of the story of what is going on under the hood. If your system is operating at 99%, it may be nothing to worry about.

    For me, the values are updating with firmware 3.60.454 and the number reported does not appear to be storing the highest level like the 'show max buffers' command would show for buffers.

    Here is the same command issued in one second intervals:

    >cpu usage
    CPU usage = 11.84% (30 sec. average = 18.70%, 30 sec. max = 41.34%)
    >cpu usage
    CPU usage = 37.81% (30 sec. average = 18.06%, 30 sec. max = 37.81%)
    >cpu usage
    CPU usage = 12.55% (30 sec. average = 18.05%, 30 sec. max = 37.81%)
    >cpu usage
    CPU usage = 31.41% (30 sec. average = 18.45%, 30 sec. max = 37.81%)
    >cpu usage
    CPU usage = 12.78% (30 sec. average = 18.44%, 30 sec. max = 37.81%)
    >cpu usage
    CPU usage = 12.60% (30 sec. average = 18.44%, 30 sec. max = 37.81%)
  • Options
    catalinocatalino Posts: 25
    Would a 99% CPU utilization indicate somewhere that you have entered an infinite loop somewhere?

    I had a situation where I found code maxing out a processor by entering a button-push infinite cycle that was executing code over and over again and locking up the processor. I fixed it by deleting the circular reference.
  • Options
    John NagyJohn Nagy Posts: 1,734
    I just updated firmware, same results here, whatever I get the first time I do CPU USAGE after a boot, that's what I get every time, forever. I just tapped 5 other systems running our software and two running other software, and get the same everywhere, always precisely the same, always very very high. It varies from boot to boot, and seems that it locks on the first use of the diagnostic.

    This from a non-CineTouch system
    >cpu usage
    CPU usage = 99.87% (30 sec. average = 99.10%, 30 sec. max = 99.87%)

    This from a CineTouch system
    cpu usage
    CPU usage = 95.12% (30 sec. average = 96.64%, 30 sec. max = 99.98%)

    Days later, load or no, these numbers are identical on a new CPU USAGE.
    Somehow, we are doing things differently. It would be great to understand this.

    UPDATE:
    I rebooted, and did the CPU USAGE during boot. It varied widely during the boot. Once the system went to idle, the usage locked at a number it now reports every time, just as before.

    With no program loaded, it shows about 8% and does vary every time I look.
    Huh.

    Update 2:
    The report is obviously false on its face.
    >cpu usage
    CPU usage = 99.99% (30 sec. average = 93.33%, 30 sec. max = 99.99%)

    It has been at 99.99% every time (several a second) for the last 5 minutes. It's mathematically impossible for the 30 second average to remain at 93.33%.
  • Options
    ericmedleyericmedley Posts: 4,177
    Hmmm.
    I just ran it on my home system and I got this

    CPU usage = 0.52% (30 sec. average = 0.86%, 30 sec. max = 2.67%)

    I'm pretty happy about it. It's a good sized program
    Pretty efficient, not bad. :)
  • Options
    viningvining Posts: 4,368
    CPU usage = 0.52% (30 sec. average = 0.86%, 30 sec. max = 2.67%)

    is everything 100% event driven with no queries unless some one is actively using the system? Even then how do you get numbers so low?

    I just used my iPad to connect to my home system and 2 remote jobs and one job was mid 50 with a 30s avg of 80 but my system and the other had all numbers in the high 90s. My system is a mess but I expected the other high average system to be much lower.
  • Options
    PhreaKPhreaK Posts: 966
    There are no magic numbers for how many Duet modules you can run on a master. There are undoubtedly limits to the number of OSGi bundles the framework can simultaneously instantiate (service id's reaching the max int / long value etc) however your memory is likely to well and truly be used long before then. There is no set your system will run at x% utilization with module i, j and k going. It is similar to asking how fast will my system run if I have y lines of code.

    Duet modules are not some magical sorcery formed by crushing newborn unicorns and injecting the resulting rainbow coloured goop into a *.jar. They are simply OSGi bundles, lumps of code whose efficiency is completed and utterly dependent on how well they are written. Unfortunately as almost every module is closed source finding that out is a matter of experimentation.

    Regarding CPU usage, we have no problems with the command reporting constantly high figures here. Its monitored and 30 second average sent back to our RMS server. This is across a number of firmware versions and a couple of years of historical data.
  • Options
    John NagyJohn Nagy Posts: 1,734
    ericmedley wrote: »
    Hmmm.
    I just ran it on my home system and I got this
    CPU usage = 0.52% (30 sec. average = 0.86%, 30 sec. max = 2.67%)
    I'm pretty happy about it. It's a good sized program
    Pretty efficient, not bad. :)

    Really amazing that you have a program that actually uses LESS CPU than no program at all.
    My test on 2 systems show about 8% CPU use idle with no program loaded at all. And with the dip#1 set to prevent anything from running... just the OS, the web server, the FTP server, duet, the clock, the I/O handlers... easily 8%. Your program somehow suspends all that and works too?

    It seems to me that the ruler is broken.
    We had come to that conclusion several years ago, and concentrate on function and performance rather than this number. But when Chris brought it up as a tool to use in diagnosis, I thought I'd ask about the clearly arbitrary results.
  • Options
    infoscreeninfoscreen Posts: 16
    CPU USAGE and DUET modules

    I'm make lot of system, but normal CPU Usage is never more than 15-20%!

    Example complete Tandberg modul with together 2-3 other DUET module:

    >cpu usage
    CPU usage = 0.41% (30 sec. average = 1.50%, 30 sec. max = 2.64%)

    USED Duet Modues:
    1 pc HitachiCPX3010N Projector
    2 pc LG monitor
    1 pc Cisco TC
    1 pc Somfy RTS
  • Options
    TechFreakTechFreak Posts: 39
    CPU Usage

    The number 1 culprit I have experienced with CPU usage is code (feedback) running in Define_Program. I had seen several systems go from 95-99% to 30% or less, just by moving ALL Feedback code into a timeline event. That was the only change made to the program, and it would consistently stay below 50%.
  • Options
    HARMAN_icraigieHARMAN_icraigie Posts: 660
    John Nagy wrote: »
    It seems to me that the ruler is broken.
    We had come to that conclusion several years ago, and concentrate on function and performance rather than this number. But when Chris brought it up as a tool to use in diagnosis, I thought I'd ask about the clearly arbitrary results.

    Agreed.

    Similar topic - any good source of information for what exactly all the different elements shown by the spy command represent?
  • Options
    jimmywjimmyw Posts: 112
    DigiMe wrote: »
    Recommended Max Duet Modules Running On An AMX Master?

    0
    Just my 0.02usd, I have jobs with huge amount of modules, code, rooms, constant streaming unsolicited feedback back, and cpu usage stays under 60%, I tested the same system with adding 2 duet modules for new components, and ui processing of page flips, feedback, everything was delayed and cpu usage was maxed 99.99, I commented the duet modules out and everything dropped back to normal. Before you say its a freak incident, this is > 5th time trying to give AMX duet modules a shot.
    ..... Why AMX..... WHY???
    ET TU AMX???

    James
  • Options
    infoscreeninfoscreen Posts: 16
    CPU USAGE - Duet modul

    Please only use JAR file. All other modules just small demo GIU!

    Same times use tom much processor time, FB is not optimal time runing!
    Communication feedback parse function is not working every time!
    EVENTS use to much times from the processor: BAD Algorithm! Not buffer parsing etc.
    If you like use other AXS files (UI, etc) all times is need check the complete program function.

    Very important point the DEFINE_PROGRAM part:

    If you use this part the program section, will running more than 6000 times/second!
    You make to FOR and other long time function, you will investigate lot of CPU times!

    Best way you make TIMELINE for all feedback!

    Good FB philosophy:
    - Not need all feedback! Only need actual view in the panel etc!
    - Not feedback every msec! Hand is pushing one button 250-350 msec. FB us enough about 400-1000 msec.

    Very important point is the STRING or COMMAND style feedback. This is use more traffic, buffers and CPU times!

    STRING and COMMAND Feedback is only need:
    - Device is come ONLINE. You need refresh the all information on the panel.
    - FB DATA is changed and PANEL is view the active page!
  • Options
    DigiMeDigiMe Posts: 59
    If there is one absolute in the known physical universe, it is this: All things are "RELATIVE".

    To say when and where feedback should be use is at best the result of cognitive bias, a concept and/or belief as the result of one's own experience and ideology. No one programmer can say the best way to write a given program and what should or should not be used.

    I am just having a little fun......ha ha.

    Either way, the point of this post was to ask a simple question. Of course, I knew it was very unlikely that any one person could provide me with an adequate answer, especially with the lack of required information on my part.

    Realistically, I knew that what I wanted to accomplish was very unlikely given that fact that I need to provide basic control of over 200+ Tandberg CODECs via an international IP infrastructure not to mention the Cisco Telepresence Codecs that can only be controlled over a SSH connection.

    Thanks all for you time....and down to DUET...
  • Options
    jjamesjjames Posts: 2,908
    From what I've heard through the grapevine, AMX has / had a tool that will put CPU usage on a graph so you could watch for any spikes during routines, etc. If it's a broken gauge, why would they have an internal tool for it?

    In a post of mine a few months back, I actually posted a screenshot comparing what a Duet & NetLinx Nuvo tuner module looked like. It was quite frightening to say the least. Keep in mind, not all Duet modules are written the same. They are the same on the outside, but deep inside their differences show. Let's just say some programmers are better than others up at AMX.
  • Options
    ericmedleyericmedley Posts: 4,177
    From the "AMX and the Uncertainty Principle department...." one of the big problems with using any CPU Meter that runs on the thing being measured is the inherent problem of how much that measurement skews the results. Perhaps the best measurement is the user experience and how they have to wait for something to happen...
  • Options
    nicholasjamesnicholasjames Posts: 154
    VxWorks Tools
    jjames wrote: »
    From what I've heard through the grapevine, AMX has / had a tool that will put CPU usage on a graph so you could watch for any spikes during routines, etc. If it's a broken gauge, why would they have an internal tool for it?

    From what I've gleaned about VxWorks and the "advanced" debug console, there is VxWorks specific software capable of interfacing with the OS at the lowest level. In comparison, the CPU USAGE and SPY commands are "helper" tools, much like SEND_STRING 0 is for most of us.
  • Options
    dspotdspot Posts: 23
    Sorry for resurrecting a quite old enough thread. I just experienced running out of duet memory several minutes ago, on a Sunday. Client replaced their 2x2 brand A wall display with brand B. I used 4 instances of duet modules for the brand A, and I thought I could just replace them with brand B ones and I'm good ready to go. Turns out that the memory ran out and not one device controllable via the virtual device. It was so simple that I just did a quick search and replace and I'm good to go.

    One other thing, not related to this, is that I wish multi-display on/off is specified in SNAPI. Or is it?
  • Options
    ericmedleyericmedley Posts: 4,177
    dspot wrote: »
    Sorry for resurrecting a quite old enough thread. I just experienced running out of duet memory several minutes ago, on a Sunday. Client replaced their 2x2 brand A wall display with brand B. I used 4 instances of duet modules for the brand A, and I thought I could just replace them with brand B ones and I'm good ready to go. Turns out that the memory ran out and not one device controllable via the virtual device. It was so simple that I just did a quick search and replace and I'm good to go.

    One other thing, not related to this, is that I wish multi-display on/off is specified in SNAPI. Or is it?

    You can allocate more memory to Duet by telnet ting into the master and typing SET DUET MEM. I typically do this from the get-go since the default setting is a little choky anyway.
  • Options
    dspotdspot Posts: 23
    ericmedley wrote: »
    You can allocate more memory to Duet by telnet ting into the master and typing SET DUET MEM. I typically do this from the get-go since the default setting is a little choky anyway.

    DUET MEM was already set to the highest possible on this master at 36M. Does the new firmware allows more memory to be allocated to duet?
  • Options
    nicholasjamesnicholasjames Posts: 154
    Is memory allocated for Duet modules that are simply residing on a Master but not being used? If so, perhaps a full clean disk is in order to clear out those unused Duet modules: "clean disk -f"
  • Options
    rrdbstudiosrrdbstudios Posts: 160
    dspot wrote: »
    DUET MEM was already set to the highest possible on this master at 36M. Does the new firmware allows more memory to be allocated to duet?

    although you can set it for 36mb, you are now taking memory resources away from your other non duet memory items.. your total memory is static, from what i understand, unless you know what your volitile and non volitile memory requirements, its not recommened to use more then half of your available memory allowcations. AMX recommend 50/50 split, dont forget sockets use up netlinx memory too..
  • Options
    AuserAuser Posts: 506
    jimmyw wrote: »
    0
    Just my 0.02usd, I have jobs with huge amount of modules, code, rooms, constant streaming unsolicited feedback back, and cpu usage stays under 60%, I tested the same system with adding 2 duet modules for new components, and ui processing of page flips, feedback, everything was delayed and cpu usage was maxed 99.99, I commented the duet modules out and everything dropped back to normal. Before you say its a freak incident, this is > 5th time trying to give AMX duet modules a shot.
    ..... Why AMX..... WHY???
    ET TU AMX???

    James

    FWIW, I added a Duet Cisco codec module to a system the other night. The system failed to operate afterwards. After updating to v4 firmware everything worked fine but now takes around 3 times as long for the control system to boot to the point where the system is operational (~6 - 8 minutes if I had to guess). Unfortunately I haven't timed the before and after boot times and this is only based on my observations.

    To answer the OP's question, based on my (bad) experiences using AMX provided Duet modules in systems, I would have to say that the max recommended Duet modules running on a master = 0.
  • Options
    viningvining Posts: 4,368
    Auser wrote: »
    I would have to say that the max recommended Duet modules running on a master = 0.
    +1, although I did use one in a project recently and it did work and it didn't bring the system to a crawl. Generally speaking though I avoid all AMX modules, especially if they're DUET.
  • Options
    HARMAN_icraigieHARMAN_icraigie Posts: 660
    I'll often have as many as 7 or 8 Duet modules running on a single NI and rarely have had issues. I've been writing all my device communication modules exclusively in Duet over the last year or and believe it has significantly cut development time. Integration of the supported device types with RMS Enterprise is seamless. I've even found Dynamic Device Discovery to be useful for video displays after discovering the generic device type.


    Just my $0.02
Sign In or Register to comment.