"Memory Available" spam
regallion
Posts: 95
Hi folks,
My current project works fine for the most part but then (seemingly) randomly throws up the following in the diag window:
At which point the app starts acting bizarrely, like I've got some memory corruption or something. There's no other messages like overrunning an array boundary or anything like that.
Anyone seen this before? Any clues as to what could be causing it?
Thanks in advance.
My current project works fine for the most part but then (seemingly) randomly throws up the following in the diag window:
Line 49 (09:38:46):: Memory Available = 34547608 <11680> Line 50 (09:38:46):: Memory Available = 34535928 <11680> Line 51 (09:38:46):: Memory Available = 34524248 <11680> Line 52 (09:38:46):: Memory Available = 34512568 <11680> Line 53 (09:38:46):: Memory Available = 34500888 <11680> Line 54 (09:38:46):: Memory Available = 34489208 <11680> Line 55 (09:38:46):: Memory Available = 34477528 <11680> Line 56 (09:38:46):: Memory Available = 34465848 <11680> Line 57 (09:38:46):: Memory Available = 34454168 <11680> Line 58 (09:38:46):: Memory Available = 34442488 <11680> Line 59 (09:38:46):: Memory Available = 34430808 <11680> Line 60 (09:38:46):: Memory Available = 34419128 <11680> Line 61 (09:38:46):: Memory Available = 34407448 <11680> Line 62 (09:38:46):: Memory Available = 34395768 <11680> Line 63 (09:38:46):: Memory Available = 34384088 <11680> Line 64 (09:38:46):: Memory Available = 34372408 <11680> Line 65 (09:38:46):: Memory Available = 34360728 <11680> Line 66 (09:38:46):: Memory Available = 34349048 <11680> Line 67 (09:38:46):: Memory Available = 34337368 <11680> Line 68 (09:38:46):: Memory Available = 34325688 <11680> Line 69 (09:38:46):: Memory Available = 34314008 <11680> Line 70 (09:38:46):: Memory Available = 34302328 <11680> Line 71 (09:38:46):: Memory Available = 34290648 <11680> Line 72 (09:38:46):: Memory Available = 34278968 <11680> Line 73 (09:38:46):: Memory Available = 34267288 <11680> Line 74 (09:38:46):: Memory Available = 34255128 <12160> Line 75 (09:38:46):: Memory Available = 34244984 <10144> Line 76 (09:38:46):: Memory Available = 34232664 <12320> Line 77 (09:38:46):: Memory Available = 34222520 <10144> Line 78 (09:38:46):: Memory Available = 34212376 <10144> Line 79 (09:38:46):: Memory Available = 34202312 <10064> Line 80 (09:38:46):: Memory Available = 34192088 <10224> Line 81 (09:38:46):: Memory Available = 34181944 <10144> Line 82 (09:38:46):: Memory Available = 34171800 <10144> Line 83 (09:38:46):: Memory Available = 34161656 <10144> Line 84 (09:38:46):: Memory Available = 34149336 <12320> Line 85 (09:38:46):: Memory Available = 34137176 <12160> Line 86 (09:38:46):: Memory Available = 34124856 <12320> Line 87 (09:38:46):: Memory Available = 34114712 <10144> Line 88 (09:38:46):: Memory Available = 34104568 <10144> Line 89 (09:38:46):: Memory Available = 34094424 <10144> Line 90 (09:38:46):: Memory Available = 34084280 <10144> Line 91 (09:38:46):: Memory Available = 34074136 <10144> Line 92 (09:38:46):: Memory Available = 34061896 <12240> Line 93 (09:38:46):: Memory Available = 34049496 <12400> Line 94 (09:38:46):: Memory Available = 34039352 <10144> Line 95 (09:38:46):: Memory Available = 34029208 <10144> Line 96 (09:38:46):: Memory Available = 34019064 <10144> Line 97 (09:38:46):: Memory Available = 34008920 <10144> Line 98 (09:38:46):: Memory Available = 33998776 <10144> Line 99 (09:38:46):: Memory Available = 33986616 <12160> Line 100 (09:38:46):: Memory Available = 33976472 <10144> Line 101 (09:38:46):: Memory Available = 33963992 <12480> Line 102 (09:38:46):: Memory Available = 33953848 <10144> Line 103 (09:38:46):: Memory Available = 33943704 <10144> Line 104 (09:38:46):: Memory Available = 33933560 <10144> Line 105 (09:38:46):: Memory Available = 33923416 <10144> Line 106 (09:38:47):: Memory Available = 33911096 <12320> Line 107 (09:38:47):: Memory Available = 33900952 <10144> Line 108 (09:38:47):: Memory Available = 33889112 <11840> Line 109 (09:38:47):: Memory Available = 33878968 <10144> Line 110 (09:38:47):: Memory Available = 33868824 <10144> Line 111 (09:38:47):: Memory Available = 33858680 <10144> Line 112 (09:38:47):: Memory Available = 33848536 <10144> Line 113 (09:38:47):: Memory Available = 33836056 <12480> Line 114 (09:38:47):: Memory Available = 33825912 <10144> Line 115 (09:38:47):: Memory Available = 33815768 <10144> Line 116 (09:38:47):: Memory Available = 33805624 <10144> Line 117 (09:38:47):: Memory Available = 33795480 <10144> Line 118 (09:38:47):: Memory Available = 33785336 <10144> Line 119 (09:38:47):: Memory Available = 33775192 <10144> Line 120 (09:38:47):: Memory Available = 33763432 <11760> Line 121 (09:38:47):: Memory Available = 33751032 <12400> Line 122 (09:38:47):: Memory Available = 33740888 <10144> Line 123 (09:38:47):: Memory Available = 33730744 <10144> Line 124 (09:38:47):: Memory Available = 33720600 <10144> Line 125 (09:38:47):: Memory Available = 33710456 <10144> Line 126 (09:38:47):: Memory Available = 33700312 <10144> Line 127 (09:38:47):: Memory Available = 33690168 <10144> Line 128 (09:38:47):: Memory Available = 33677848 <12320> Line 129 (09:38:47):: Memory Available = 33667704 <10144> Line 130 (09:38:47):: Memory Available = 33655464 <12240> Line 131 (09:38:47):: Memory Available = 33643224 <12240> Line 132 (09:38:47):: Memory Available = 33633080 <10144> Line 133 (09:38:47):: Memory Available = 33622936 <10144> Line 134 (09:38:47):: Memory Available = 33610456 <12480> Line 135 (09:38:47):: Memory Available = 33600312 <10144> Line 136 (09:38:47):: Memory Available = 33590168 <10144> Line 137 (09:38:47):: Memory Available = 33577688 <12480> Line 138 (09:38:47):: Memory Available = 33567544 <10144> Line 139 (09:38:47):: Memory Available = 33557400 <10144> Line 140 (09:38:47):: Memory Available = 33547256 <10144> Line 141 (09:38:47):: Memory Available = 33537192 <10064> Line 142 (09:38:47):: Memory Available = 33525272 <11920> Line 143 (09:38:47):: Memory Available = 33515128 <10144> Line 144 (09:38:47):: Memory Available = 33502808 <12320> Line 145 (09:38:47):: Memory Available = 33492664 <10144> Line 146 (09:38:47):: Memory Available = 33482520 <10144> Line 147 (09:38:47):: Memory Available = 33472376 <10144> Line 148 (09:38:47):: Memory Available = 33462232 <10144> Line 149 (09:38:47):: Memory Available = 33449752 <12480> Line 150 (09:38:47):: Memory Available = 33439608 <10144> Line 151 (09:38:47):: Memory Available = 33427528 <12080> Line 152 (09:38:47):: Memory Available = 33417464 <10064> Line 153 (09:38:47):: Memory Available = 33407320 <10144> Line 154 (09:38:47):: Memory Available = 33397176 <10144> Line 155 (09:38:47):: Memory Available = 33387032 <10144> Line 156 (09:38:47):: Memory Available = 33376888 <10144> Line 157 (09:38:47):: Memory Available = 33364568 <12320> Line 158 (09:38:47):: Memory Available = 33354504 <10064> Line 159 (09:38:47):: Memory Available = 33344440 <10064> Line 160 (09:38:47):: Memory Available = 33334296 <10144> Line 161 (09:38:47):: Memory Available = 33324152 <10144> Line 162 (09:38:47):: Memory Available = 33314008 <10144> Line 163 (09:38:47):: Memory Available = 33301848 <12160> Line 164 (09:38:47):: Memory Available = 33289528 <12320> Line 165 (09:38:47):: Memory Available = 33277208 <12320> Line 166 (09:38:47):: Memory Available = 33267064 <10144> Line 167 (09:38:47):: Memory Available = 33256920 <10144> Line 168 (09:38:47):: Memory Available = 33246776 <10144> Line 169 (09:38:47):: Memory Available = 33236632 <10144> Line 170 (09:38:47):: Memory Available = 33226488 <10144> Line 171 (09:38:48):: Memory Available = 33214248 <12240> Line 172 (09:38:48):: Memory Available = 33202568 <11680> Line 173 (09:38:48):: Memory Available = 33190888 <11680> Line 174 (09:38:48):: Memory Available = 33179208 <11680> Line 175 (09:38:48):: Memory Available = 33167528 <11680> Line 176 (09:38:49):: Memory Available = 33155848 <11680> Line 177 (09:38:49):: Memory Available = 33144168 <11680> Line 178 (09:38:49):: Memory Available = 33131928 <12240> Line 179 (09:38:49):: Memory Available = 33121784 <10144> Line 180 (09:38:49):: Memory Available = 33109544 <12240>
At which point the app starts acting bizarrely, like I've got some memory corruption or something. There's no other messages like overrunning an array boundary or anything like that.
Anyone seen this before? Any clues as to what could be causing it?
Thanks in advance.
0
Comments
Along the lines of the following (though likely to be much harder to find when embedded in code).
define_function GobbleMemory() { stack_var char _cTastyMemory[11680] GobbleMemory() }If you issue a few "show mem" commands from the console when this occurs, can you see the available memory reducing and eventually hit a negative number (at which point the program will act bizarrely as you noted)?
I would personally start looking for any arrays that have a dimension with size of 10000 in your code just looking at the way the memory is being used and then see what is calling the code where they're being instantiated.
What exactly causes one of those messages to appear? Does it happen every time an array is created?
The messages appear whenever the available memory is reduced, the question is what reduces the available memory.
I just had a play with the following code and found something very interesting:
PROGRAM_NAME='Foo' DEFINE_VARIABLE volatile integer nMemoryAssignmentCount = 0 volatile integer nDelayCounter = 0 define_function GobbleMemory() { //stack_var char _cTastyMemory[10000] nMemoryAssignmentCount++ send_string 0, "'GobbleMemory()',$0D,$0A" nDelayCounter = 0 while(nDelayCounter < 10000) nDelayCounter++ if(nMemoryAssignmentCount <= 10) GobbleMemory() } DEFINE_EVENT data_event[5001:9:0] { online: { wait 50 { send_string 0, "'Event: Dinner.IsServed = TRUE',$0D,$0A" GobbleMemory() } } }Whether or not I had the stack var commented out, I received the following output:
So it would seem that memory is allocated when a function is called but not when stack memory is allocated. Hmm. (Note that I ensured that the variable was assigned to and used in a conditional clause in one version of the code to prove that it was actually being created and wasn't being optimised out by the compiler).
Note that when the memory allocation changes it seems to be updated ~ every second in general. The interesting thing about the output you've posted is that the memory allocation is updated many times per second. I think the only time I ever see that is at boot time (when there's lots of stuff going on in Java land).
I can't offer too many suggestions based on the results of the tests I just ran. Out of curiosity are you using Duet modules and how often does the problem occur?
So far, I've failed to reliably replicate the phenomenon. The project is close to 15,000 lines of code now so nailing this sucker is painful!
Still - at least I know it has something to do with memory allocation and maybe I do have some code somewhere that's doing some mad looping or something.
All I can do at this stage is pepper the code with SEND_STRING 0's in an attempt to track it down.
Thanks for your time so far, if I figure it out I'll let you know.
I'd say Auser is on the money, chances are there is a condition happening that is causing infinite / larger than you're likely to want recursion.
This is certainly the case in other platforms, I was just surprised that allocating 10k chunks wasn't causing the pool size to increase. Of course what I hadn't considered was the fact that there's a lot of one off consumers off stack memory space at boot time and I din't hit the limit of the pool in 10 - 20 iterations. The program completes in such a short time after the control system boots that any garbage collection/stack memory pool management processes which may exist won't have a chance to release unallocated memory from the pool.
It would be interesting to instantiate a one off stack var in my test code that's on the order of MB's before starting the iteration in order to ensure that all pool space is consumed before starting to instatiate the10kB stack var...