Suggestions for new NetLinx compiler features/enhancements
jeffaco
Posts: 121
Based on another thread, I thought it would be useful to raise some suggestions for new NetLinx compiler features and enhancements. It might be useful to share our ideas with one another.
I'll start, since I'm starting the topic! ;-)
1) By far, by biggest wish: Some sort of procedural callable interface to modules (sort of like a C++ class definition or something like that). Right now, if I want to get information from a module, the only way I can do it is to send a message, and then wait for the response.
I can have a callable interface of sorts (i.e. I can have an include file that defines an interface and sends a message to the module). But if I want a response (return value or whatever), that becomes somewhat clumsy since I now have to stop my thread of execution and wait for the response (essentially creating a state machine).
2) More careful error reporting, particularly with line numbers. At times, the line numbers seem to get "off by one" (particularly when including other files). End result: When the compiler gives an error, it refers to a line number that doesn't have a problem.
3) More robust error reporting. An error is an error. Complex, or long involved code, shouldn't keep errors from appearing.
4) TCP/IP improvements:
a) For example, if I'm opening a socket to some other machine (and waiting for a timeout because the target isn't responding), I can't stop that (i.e. executing a close won't cancel an existing open).
b) Some way to dynamically choose port numbers for socket communications. Right now (near as I can tell), they must be hard coded. This creates issues for modules that might want to communicate to 'n' other places.
Those are the biggies that come to mind immediately. Other thoughts?
I'll start, since I'm starting the topic! ;-)
1) By far, by biggest wish: Some sort of procedural callable interface to modules (sort of like a C++ class definition or something like that). Right now, if I want to get information from a module, the only way I can do it is to send a message, and then wait for the response.
I can have a callable interface of sorts (i.e. I can have an include file that defines an interface and sends a message to the module). But if I want a response (return value or whatever), that becomes somewhat clumsy since I now have to stop my thread of execution and wait for the response (essentially creating a state machine).
2) More careful error reporting, particularly with line numbers. At times, the line numbers seem to get "off by one" (particularly when including other files). End result: When the compiler gives an error, it refers to a line number that doesn't have a problem.
3) More robust error reporting. An error is an error. Complex, or long involved code, shouldn't keep errors from appearing.
4) TCP/IP improvements:
a) For example, if I'm opening a socket to some other machine (and waiting for a timeout because the target isn't responding), I can't stop that (i.e. executing a close won't cancel an existing open).
b) Some way to dynamically choose port numbers for socket communications. Right now (near as I can tell), they must be hard coded. This creates issues for modules that might want to communicate to 'n' other places.
Those are the biggies that come to mind immediately. Other thoughts?
0
Comments
(a) A way to read a level like we can read channel states. I find it cumbersome to have to define a variable for that when obviously (at least on a single system), the system knows LEVEL.VALUE at all times since it won't send a level event with the same value than the last one. I'd like to have access to this "last value" from the code.
(b) Automatic stripping of unused code, in particular functions.
(c) Better support for 2 way exchanges, i.e. item 1 of the previous post extended to all 2 way devices: ethernet, serial, virtual devices.
Basically not having to do state machines for stuff like "send X, wait to receive Y", along with the ability to lock a queue (i.e. stop processing of events until released). Having this for virtual devices is a way to solve the problem post 1 tries to fix.
That means moving away from strict user-code based cooperative multitasking to compiler helped cooperative multitasking or obviously to full multitasking (i.e. threads, locking calls and the whole shebang).
But the absolute top priority is to fix the line numbers in the compiler reports. Maybe I am doing something wrong, but it NEVER gives me the correct line number.
Fred
What do you mean by: "Automatic stripping of unused code, in particular functions."
I actually got into a long discussion with a friend about issue number 1 in my original post (procedural callable interface for modules).
Other languages that are event based (similar to NetLinx) have a construct to wait for an event *INLINE*. For example, you might do something like (translated, more or less, to NetLinx):
SEND_COMMAND vdModule, "$OPCODE_GETDATA";
GET_RESPONSE [vdModule] {
STRING: // This block executes on string response
ERROR: // This block executes on error
WAIT [50]: // This block executes on timeout (no response)
}
(Note: I was careful here. In this example, commands are sent via SEND_COMMAND, and responses come back as STRINGS. This is important to avoid the need for multiple devices or something.)
If this were provided for all devices (Ethernet, virtual devices, etc), such that the code continues inline, but other events could execute while waiting, that would be "good enough" for me. This would be enough so that I can have, in an include file, a true "callable interface" defined where I can get responses just by making a subroutine call, which is what I ultimately want.
By automatic stripping, I mean removing code which is not used. Typically we all have a 'Utils.axi' or equivalent that contains a lot of often used constants and functions. I include it in every program/module. But it may happen you only use 2 or 3 functions out of the 30 or so Utils may contain. In this case I would like the compiler to be "smart" and get rid of the compiled code it created for them. Because NetLinx is entirely static (with regards to code execution, i.e. there is no inheritance or "jump to pointer" thing), it is entirely possible to determine the list of functions used. It could even issue a warning (used to do that in axcess for variables).
Performance wise I don't think it matters much once the code is on the master and it would slow down compilation, but the code would be smaller and that can't be a bad thing, it's cleaner and should not be that hard to implement (IMHO).
Fred
Fred - Take a look at the Compiler Directive #IF_DEFINED in the help file and see if that gets you where you want to go.
Joe
Yes, regarding your sample code, I agree. But then you need locking...
What if your 'vdModule' needs to talk to a serial device to give you a reply to '$OPCODE_GETDATA' ? Inside vdModule, the code looks like:
DATA_EVENT[me]
{
COMMAND:
{
SWITCH(Data.Text):
{
CASE $OPCODE_GETDATA:
{
SEND_COMMAND RS232Port, "$something";
GET_RESPONSE [RS232Port]
{
STRING: SEND_STRING Data.?CALLER?
ERROR: // This block executes on error
WAIT [50]: // This block executes on timeout (no response)
}
}
}
}
}
One issue is Data.?Caller?. Today ALL send_XXX have the master as the caller. But that can be fixed somewhat easily with new SEND_TRACEABLE_STRING or whatever that have a device as parameter to reply to.
But the other issue is that today, while you are waiting for the reply of the RS232 device inside the module, you *CAN* get events for "new" $OPCODE_GETDATA requests. That means you have to write the handling code as re-entrant, which is a pain. What you want it to have the calling code wait for the reply AND the called code handle one request at a time, that means locking (queuing) any SEND_COMMANDs to vdModule until the request is handled.
What you propose is what I was referring to as "compiler assisted cooperative multitasking", that it, the compiler proposes some functions to make it easier.
In my code, I use a locking mechanism that does exactly that. It can't refuse events, but it sends a "busy" reply immediatly to the caller. I also have code that implements the waiting until no more busy to resend the command...
Fred
Actually, writing reentrant code isn't a big deal to me. It's easy to write reentrant code once you understand the rules, but then most AMX folks aren't programmers (as such), and concepts like multithreaded code, reentrancy, pointers, etc is probably beyond many of the folks that program NetLinx gear. But I digress.
I understand the issues you raise. These are issues, and these would need to be worked out. What I proposed is the "nirvana" (i.e. some sort of interface to do inline send/receive without dealing with multistate machines).
Frankly, I'd be happy with a procedural interface to modules, where (as long as the module doesn't need to send data to some physical device) I can get the response inline, immediately.
I can define function in mainline, and I get that.
I can define a function in an include file, and I get that.
But if it makes logical sense to define that function in a module, then I lose that. That's unfortunate, and complicates code quite a bit if I *NEED* to do that.
A guess pointer are referring to the same thing as in C++(&, *)
Good look guys and keep up the good work.
Thanks to the people who brought to life this Forum.
I'm not convinced that pointer support is a good idea for a language like NetLinx.
Certainly, there have been plenty of situations where it would have been useful. The problem (without pointer support) is that you need to size things for maximum size, which isn't generally the size you want or need (particularly problematic in modules, where the module can be run from many different places). In this case, I presume you mean "pointer support" to also include some form of dynamic memory allocation as well.
There have also been situations for me where just passing a pointer to an already allocated array would be useful as well (although I generally get by without this).
But think about what pointer support brings along as baggage:
Ability to crash your application in about 1 second flat,
Depending on how the O/S itself is protected (or lack thereof; I'm not sure what VxWorks offers in this area, or what the NetLinx processor board supports), you can crash the O/S itself in about the same amount of time. Look at Win98, or WinME; it was trivial to have a one or two line program bring down one of these operating systems. Because of this, most people felt that "Windows is unreliable", where in the majority of the cases, it was actually the applications that were unreliable and the operating system was permitting the application to do "bad stuff".
There would be serious debugging issues, particularly if errant pointer handling is very state-specific (I can hear the complaints now: "Most times my program runs fine, but then for seemingly no reason, the NetLinx processor locks up and totally stops responding") ...
Considering that most of the NetLinx installers and programmers aren't really programmers by profession, I suspect this is just a little to much to bring to the table, and could quite possibly overwhelm tecnical support in dealing with issues that are quite difficult to diagnose and find, particularly in long, complex programs.
Many languages today (i.e. PERL, VB and countless others) don't give this level of flexability, but instead give dymanic memory allocation with rich built in garbage collection. For something like NetLinx, I suspect this would be the better solution, and gives nearly the same flexibility with much better overall reliability in the face of programmer errors.
The Netlinx product should be a rock solid piece of soft.
I don?t want to say that programmers out there are not capable of handling this (pointers) but playing with pointers it is not a joke (like you say ?Ability to crash your application in about 1 second flat? and time spend on finding the problem)
Good lock out there.
After you have defined a function, i would like Netlinx Studio to show a popup box giving a hint to what the parameters are.
ie in your definations you have:
DEFINE_CALL 'Projector Function'(Projector_Number, Projector_Mode)
then further down in your events you go to call it:
CALL 'Projector Function' <
A little popup box will appear now, showing "Projector Function (Projector_Number, Projector_Mode), reminding you which parameter you have to enter first.
You could then take that a step further (and this will require a compiler change) and allow the use of enumerators.
ie..
Enum Projector_Mode
Projector_Off
Projector_On
Projector_Video
Projector _VGA
Define_Call 'Projector Function'(Projector_Number, Mode AS Projector_Mode)
This will then only allow you to enter the values described in Projector_Mode, into the Mode parameter of your Function.
This is one of the things that I missed from going to AMX programming from Visual Basic, as it allows you to write air-tight little modules, with everything already defined.
It would also make calling functions from inside different modules easier, as you would know what values they are expecting, without having to open up the code and check.
Turn off Autocomplete when typing Strings or Comments..
dev devAllTouchPanels[] = {dev_tp_computer, dev_tp_kitchen, dev_tp_masterbed, dev_tp_theater} devchan dcLivingRoomHalogens[] = {{dev_tp_computer,102}, {dev_tp_kitchen,102}, {dev_tp_masterbed,102}, {dev_tp_theater,102}} devchan dcLivingRoomLamps[] = {{dev_tp_computer,103}, {dev_tp_kitchen,103}, {dev_tp_masterbed,103}, {dev_tp_theater,103}} devchan dcKitchenPanRack[] = {{dev_tp_computer,104}, {dev_tp_kitchen,104}, {dev_tp_masterbed,104}, {dev_tp_theater,104}} devchan dcKitchenMain[] = {{dev_tp_computer,105}, {dev_tp_kitchen,105}, {dev_tp_masterbed,105}, {dev_tp_theater,105}} devchan dcKitchenSink[] = {{dev_tp_computer,106}, {dev_tp_kitchen,106}, {dev_tp_masterbed,106}, {dev_tp_theater,106}} devchan dcKitchenAccent[] = {{dev_tp_computer,107}, {dev_tp_kitchen,107}, {dev_tp_masterbed,107}, {dev_tp_theater,107}} ... etc, for dozens of channels..This could be simplified with:dev devAllTouchPanels[] = {dev_tp_computer, dev_tp_kitchen, dev_tp_masterbed, dev_tp_theater} devchan dcLivingRoomHalogens = {dev_AllTouchPanels,102} devchan dcLivingRoomLamps = {dev_AllTouchPanels,103} devchan dcKitchenPanRack = {dev_AllTouchPanels,104} devchan dcKitchenMain = {dev_AllTouchPanels,105} devchan dcKitchenSink = {dev_AllTouchPanels,106} devchan dcKitchenAccent = {dev_AllTouchPanels,107}Would make using devchan arrays a lot easier.Couldn't of you used a Virtual Touchpanel to do that:
ie..
DEFINE_DEVICE
vdAllTP = 33001
DEFINE_VARIABLE
devchan dcLivingRoomHalogens = {vdAllTP,102}
devchan dcLivingRoomLamps = {vdAllTP,103}
devchan dcKitchenPanRack = {vdAllTP, 104}
devchan dcKitchenMain = {vdAllTP,105}
devchan dcKitchenSink = {vdAllTP,106}
devchan dcKitchenAccent = {vdAllTP,107}
DATA_EVENT[ will probably have to do one for each TP ]:
{
ONLINE:
{
COMBINE vdALLTP, 1stTP, 2ndTP, etc...
}
}
Pretty sure that would do the same thing your asking..
...I liked the old autocomplete better. the new version has two big problems for me.
First, you can't make the pop-up go away with the arrow keys. This makes sense with up and down, but if you make a typo now, but the autocomplete kicks in, backspace and the left and right arrows don't make it go away. I find it very irritating to have to hit escape before I can correct a typo.
The other thing is if you are correcting an item an it suggests your correction properly, it inserts it in the middle of what you are typing instead of replacing the entire word, so you have to go back and edit it again.
Both of these things make a time-saving feature take more time to use than if it wasn't there.
Define_Module ....
#include xyz.axi
NetLinx Studio will automatically put the code in appropriate location for you.
what would be really nice is a separate command line compiler for Netlinx a-la accessx. I use accessx within editplus2 as well as on linux and it is a lot nicer having choice on IDE. I have written my own syntax files etc so code highlighting is done automagically. A command line Netlinx compiler would allow people on Mac and *nix boxes to compile for netlinx as well.
I agree. I would very much like a command line compiler for all the same reasons. I like using a different editor, and I usually only use Studio to compile what I've done somewhere else.
I'd like to be able to pass user defined structure types as parameters to modules. While it is possible to pass all the data/variables as intrinsic types, it defeats the purpose of defining your own structures in a way that resembles the process at hand. This would (at least in my case) greatly reduce the number of parameters I currently have to pass into modules.
Joe,
Somehow I only see your post now... But yes, that's what I am doing but it is a pain to do manually with 4 includes with 30 functions each... For example I have a DateTime type that I use often which offers a ton of functions like AddSecond, AddDay, AddWeeks, Compare, etc... In every single program/module I use it I only use 2 or 3 functions...
The compiler KNOWS the complete list of functions. A basic implementation is to remove any function that is never called by any other. A more clever one is to do so recursively.
Fred
Also using structures as return values for functions...
Fred
It seems everybody wants the NetLinx language to evolve towards a more "matured" language/compiler like C++, Modula-2 or Perl.
- Separate the compiler from Studio
- Make it stronger:
-- Stronger and better error reporting (line numbers...)
-- Better optimization
- Better user types support:
-- Support for enums, smarter arrays
-- Support for user types everywhere we can use an intrisic type
- Support for modern notions:
-- Pointers or at least dynamic memory management
-- Modules with a real procedural interface
We have comparatively very few "minor" improvements compared to the "major" ones above:
- Better IP sockets (cancellable, dynamic)
- Reading level values
I did not compile the suggestions related to the Studio Editor, as the thread was about the compiler only.
Fred
Fred,
What do you mean by 'dynamic'? Does this refer to UDP or TCP or Both?
Here's something that would be nice to see in Studio. Many times the master controller is located in another room and I am connected to it via a long cable. During troubleshooting, it really helps to see the front of the master to see if the Tx and Rx LEDs are lighting or not. I know you can watch the same sort of thing in the notifications tab, but just having a visual of the master would be very beneficial. I'm sure there is a way for Studio to tell which type of master you are connected to (NXI, NI series, etc.).
As far as user defined structures as return types for functions, YEAH, absolutely huge. Good catch.
Since I was the one that asked for the TCP/IP and UDP improvements, I'll respond. Fred (very useful, thanks) summarized other folks suggestions.
I do a lot with TCP/IP (both TCP and UDP). In fact, I was the one that wrote the basis for the i!TimeManager module and submitted it to AMX, who made a few improvements and released it. You can see the original source code at:
http://cvs.sourceforge.net/viewcvs.py/netlinx-modules/NetLinx-Modules/TimeSync/
Anyway, the problem is two fold:
1) With TCP/IP: Once you begin an open operation, you can't cancel it. You need to wait for the timeout.
2) With both TCP/IP and UDP: You must choose the port to use statically to insure there's conflict with other ports. If you have a lot of different modules or code segments that use TCP, that's a pain. Or worse, if you have a module that can create a dymanic number of connections based on context, that's very difficult to manage.
So, basically, I don't care much if it's TCP or UDP. I'd like some piece of code to find the next available port number and use it. When done, it can close it. If code needs to create a connection to five different hosts at the same time, so be it. I can manage an array of devices (or something) for multiple connections and have an event handler for the array of devices to handle the data events.
There's currently no construct to find the next available port number, so this isn't possible today.
BTW, this has a side benefit: Modules that use the network would no longer need to take in a virtual device as a parameter to control what port was used. So modules that used the network would be simplified.
Hope this clarifies,
-- Jeff
"1) With TCP/IP: Once you begin an open operation, you can't cancel it. You need to wait for the timeout."
I see. Yes, that's very valid.
"2) With both TCP/IP and UDP: You must choose the port to use statically to insure there's conflict with other ports. If you have a lot of different modules or code segments that use TCP, that's a pain. Or worse, if you have a module that can create a dymanic number of connections based on context, that's very difficult to manage."
a. I asked which protocol only because originally, UDP (as provided by AMX) was only good for one IP address per connection, even though it is a connectionless protocol. They've since provided a workaround for that which allows you to 'dynamically' change the IP address you wish to send a data packet to.
b. Yes, I see what you mean about having to chose local ports statically, but (and this goes without saying) since NetLinx is event driven, and these transmission events are based upon devices, the idea of changing devices in mid stream seems a little foreign. I'm not saying your request is absurd, just a little out of the box.
"BTW, this has a side benefit: Modules that use the network would no longer need to take in a virtual device as a parameter to control what port was used. So modules that used the network would be simplified."
Well, I guess that depends how you write your modules. I wrote a module that does nothing but ensure a valid ethernet connection. It requires a virtual device, the port device (0:p#:0), the ethernet parameters (as a character string), a debug channel, and an online channel. I use this module in conjunction with any other Comm Module I write. All my Comm modules have a small piece of code in them that gets triggered from a Device ONLINE event. It looks at the device.Number to see if it is non-zero. If it is, it issues the SET BAUD command, waits 3/10ths of a second, and turns on an ONLINE channel on the virtual device. If the device.Number is zero, it only waits 30 milliseconds and then turns on the ONLINE channel on the virtual device.
What this does is allow me to use Ethernet or Physical 232 port at will, and still use the same virtual device and physical device for each. Once the Ethernet module makes a connection, the Comm module will get the online event just like it would for a 232 port, everything else is the same.
Not that this couldn't be done another way, but the Ethernet module also allows for changing the connection by changing the IP settings passed into it, and sending a reconnect command. I realize this doesn't address everything you've discussed, but it comes pretty close, and it eliminates the redundancy of a fair amount of code for the ethernet connection inside the Comm module.
There are other benefits to doing this as well, but this post is long enough.
I was actually trying only to summarize and categorize the ideas... Both adjectives next to "Better IP sockets" were an attempt (failed, apparently :-) ) at summarizing Jeff's more complete description in the first post, quoted below for your convenience.
Fred
Jeff means that having to allocate a port NUMBER which is not in conflict is a pain. With ethernet you do not care if port 4 is for 'www.smurf.com' and 9 for 'www.rabbit.com', as long as you know which is which. You can swap the port numbers and everything still works. Not true with RS232 or IR; there is a strong physical reality behind it that renders the effort valuable (a port means a given wire and you need to make sure your program knows which wire it is talking about)
On the other hand, the way IP ports are managed today is COHERENT with the rest of the language, and coherence is a good thing.
I guess Jeff would like something like:
DEFINE_DEVICE
DEV my_port1 = NEXT_LOCAL_PORT;
where NEXT_LOCAL_PORT would be a new language keyword that gives my_port1 whatever id is available. To meet Jeff's requirement you would need to be able to call NEXT_LOCAL_PORT anywhere in the program (not only in DEFINE_DEVICE) and it would return the next available (non open) ethernet port.
Fred