This project has moved and is read-only. For the latest updates, please go here.

Memory Usage - affect of lagtime, TrackLatestMeasurements, etc

Apr 18, 2013 at 7:46 PM
In using various adapters, including the PI Input adapter, and many DynamicCalculation adapters using calculated measurements, I've had to use very large lagtimes, up to 50 seconds, to get 'reasonable' results. All seems well at this time, but memory use has increased to over 3G. Is this expected from these adapters, lagtimes, or other parameters?
Apr 18, 2013 at 8:09 PM
Using such large lag-times when synchrophasor data is flowing into an adapter will definitely engage very large amounts of memory.

Which measurements are you waiting on? SCADA values from PI, for example?

There may be a better way if I can understand your requirements a little better...

Apr 18, 2013 at 8:55 PM
I was advised by Ryan McCoy that MISO had to use very large lag times (30 seconds or more) to allow all PI data to arrive for comparisons via the DynamicCalculator. We intend to use the PI data for several purposes:
1. A simple comparison of MW, MVAR values (from PowerCalculation adapter) against SCADA and SE values - also MVA values calculated in yet another DynamicCalculation from the MW,MVAR values.
2. Most important, we pick two reference phase angles, one from PMU and one from PI, hopefully for the same measurement. For each value, calculate the difference between this reference and the other phase angles in the same space (PMU reference diff PMU Phase angles, and PI reference vs PI phase angle. Then, we compare the difference values from PMU to PI on the same measurement; this difference should be small.

Apr 18, 2013 at 9:13 PM
Edited Apr 18, 2013 at 9:51 PM
If you are only interested in the results when a slower moving piece of data actually arrives, you should be able to change the "framesPerSecond" connection string parameter to 1 such that the system will only allocate 1 bucket for every second of lag-time (instead of 30 times the number of data buckets per each second of lag-time). This will cause the calculation to be processed once per second and should greatly reduce required memory footprint and CPU loading.

In cases where you are receiving synchrophasor data 30-times per second but setting the frame processing rate to 1, down-sampling will occur. There are options on best value to use by specifying the "downsamplingMethod" connection string parameter, at least if it matters greatly, see here:

The concentrator algorithm in the openPDC is designed to sort measurements being transmitted in real-time for data being sent at rates of at least 1 sample per second. Slower rates (e.g., once every few seconds) are not supported since sorting data at these speeds would be trivial. There is no defined maximum number of supported samples per second, but CPU utilization (and memory) will increase as the measurement volume and frame rate increase.

For custom action adapters that sort data at slower speeds (i.e., slower than 1-frame per second) you have the option of using the FacileActionAdapterBase. This just receives and processes data as it arrives (no sorting / no lag-time queue) allowing the user to decide when processing can occur. Note that the facile action adapter type still specifies the lag/lead time values, but they are only used as a time-range validation for temporal measurements when the "TrackLatestestMeasurements" function is enabled.