This project has moved. For the latest updates, please go here.

Creating Subscription Authorization Request

Developer
Apr 23, 2013 at 6:04 PM
I am attempting to set up a gateway style connection between two servers each running an instance of the latest build of the openPDC. The two machines each have identical read-only databases where the openPDC configuration is stored. To edit this. I must do so from another server and the updates are log-shipped to the two servers of interest. (Not a great implementation but out of my control)

I wish to create a gateway style connection between the two servers. First I must, from the server that I wish to be Subscriber, click on 'Gateway' - 'Susbcription' - 'Create Authorization Request'? At least that is how I understand it. However, because the DB on that server is read-only, I cannot create the associated Data Subscriber Adapter (by checking the check box). If I model this on the third server (where the only writable DB sits) I am warned that I should perform this operation on the server where the service is running. I believe there must be something to this because when I model this on the writable DB I get a message stating:

"Unable to notify service about updated crypto cache: Application is currently disconnected from service"

Of course, I am on another machine. What I am trying to figure out is if I should ignore these warnings or try to find some work-around. Because I can't model it directly on the desired server, I see two options...
  1. Create the Authorization Request on the desired server but do not create the Data Subscriber Adapter (by not checking the check box) and subsequently create (using the writable DB on the third server) a Data Subscriber Adapter with necessary configurations (although I dont know what those would be). Then proceed with modeling my pub/sub connection.
  2. Create the Authorization Request on the third server with the writable DB and change the Valid IP Address field to the IP address of the Publisher server with the read-only DB with the default port number for the ExternalDataPublisher and somehow hope that the service gets notified about updated crypto cache some other way.
Either, how can I model the Data Subscriber Adapter properly to accomplish scenario 1 or how can I make sure the service on my Subscriber server is notified of the update of the crypto cache?

Thoughts? Suggestions?

Thanks!!!!!!

Kevin
Coordinator
Apr 27, 2013 at 4:42 PM
Extremely complex problem - simple fix, give all openPDC instances R/W access to their own database - but since that's not likely an option, let's make sure you actually need an "authorized" subscription.

Unless you needing to validate the identity of the remote subscription or enable data encryption, you can use a simple subscription between two gateway instances which would greatly simplify that task at hand.

Do you need to the authorization or will something more simple work?

Thanks,
Ritchie
Developer
Apr 27, 2013 at 4:51 PM
No authorization needed :) What I need is to be able to define different privileges for each of my connections. I would like to send positive sequence to one server and ABC to another. I have come to the assumption I can't do that with a simple subscription. But maybe with an appropriate filter? The filter we thought was filtering measurements wasnt. The only way we found to get ABC measurements away from the Positive sequence subscriber was by changing the Subscribed check box. Not sure here...

Thanks,

Kevin
Coordinator
Apr 27, 2013 at 4:59 PM
Ignoring "privileges" for the moment (since that would require an authorized subscription) - you can always use a filter expression on the subscribers to just get positive sequence, for example:
    outputMeasurements={FILTER ActiveMeasurements WHERE SignalType LIKE '*PH*' AND Phase= '+' ORDER BY PhasorID}
This expression would filter measurements for subscription for just voltages and currents for both angles and magnitudes. Additionally, by sorting by the phasor ID the angles and magnitudes would be paired together such that you would know which angle went with which magnitude.

Would something like this not work?

Ritchie
Developer
Apr 27, 2013 at 5:04 PM
Yes! I think it would :) and I think this is what was originally intended. However, I think the filter expression used must have been flawed because it read like it should have been sending only positive sequence and F, dfdt, status etc but was actually letting everything through. I don't have the filter expression offhand but this looks to be like a great place to investigate.

Thanks for the sanity check!

Kevin
Developer
Apr 29, 2013 at 2:17 PM
I have investigated our filtering expression for our simple subscriptions and here is what I have found:

Our original filtering expression was the following:
outputMeasurements=FILTER ActiveMeasurements WHERE (Phase IS NULL) or (Phase = '+');
This was inneffective at removing the ABC measurements from the stream. I thought it might be because of the missing curly brackets so I tested the following filter expression:
outputMeasurements={FILTER ActiveMeasurements WHERE (Phase IS NULL) or (Phase = '+')};
I had similar results. ABC measurements still remain. A wierd side effect, though, our DynamicCalculator adapters (what my predecessor originally used for naming convention translation) for our first live substation ceased to function (probably didn't get measurements - but I dont know why) Anywho...

I decided to try a more direct filter expression, hopefully leaving nothing to chance:
outputMeasurements={FILTER ActiveMeasurements WHERE (SignalType LIKE '*PH*' AND Phase='+') OR (SignalType='FREQ') OR (SignalType='DFDT') OR (SignalType='FLAG') OR (SignalType='DIGI')}
This also yielded ABC measurements in the output measurements of the subscription. Back to square one.

I am assuming that I have made some sort of fundamental mistake in my filter expression. How does it look?

Thanks!!

Kevin
Coordinator
Apr 29, 2013 at 2:40 PM
Hi Kevin,

The query looks fine to me. FYI, you can verify your filter expression by running a similar query against your database. Assuming your node ID is "186C8E88-D86A-46AD-926B-C10905F37767":
SELECT * FROM ActiveMeasurement WHERE ((SignalType LIKE '%PH%' AND Phase='+') OR (SignalType='FREQ') OR (SignalType='DFDT') OR (SignalType='FLAG') OR (SignalType='DIGI')) AND (NodeID = '186C8E88-D86A-46AD-926B-C10905F37767')
Also, you can determine which measurements your node will be subscribing to with the following query.
SELECT * FROM ActiveMeasurement WHERE ((SignalType LIKE '%PH%' AND Phase='+') OR (SignalType='FREQ') OR (SignalType='DFDT') OR (SignalType='FLAG') OR (SignalType='DIGI') OR (Subscribed <> 0)) AND (NodeID = '186C8E88-D86A-46AD-926B-C10905F37767')
The Subscribed flag is used in conjunction with the outputMeasurements filter expression, so that may be messing with the results.

Thanks,
Stephen
Developer
Apr 29, 2013 at 2:58 PM
Thanks for the suggestion, Stephen.

Could you elaborate on how the Subscribed flag is used in conjunction with the otuputMeasurements filter expression? I imagine it is as you have shown above but I just wanted to clarify...

My only definitive observation thus far is that the measurement is an output measurement of the simple subscription if the Subscribed flag is true. if it is false, then it is not an output measurement. It seems to be almost irrelevant of the filter expression. Could this be caused by having all of our nodes measurements under the same table? I am trying to understand all of the mechanisms at work here...

Thanks so much for your help :)

Kevin
Coordinator
Apr 29, 2013 at 3:08 PM
Yes, the following code from DataSubscriber.Initialize should help to explain...
                // If active measurements are defined, attempt to defined desired subscription points from there
                if (DataSource != null && (object)DataSource.Tables != null && DataSource.Tables.Contains("ActiveMeasurements"))
                {
                    try
                    {
                        // Filter to points associated with this subscriber that have been requested for subscription, are enabled and not owned locally
                        DataRow[] filteredRows = DataSource.Tables["ActiveMeasurements"].Select("Subscribed <> 0");
                        List<IMeasurement> subscribedMeasurements = new List<IMeasurement>();
                        MeasurementKey key;
                        Guid signalID;

                        foreach (DataRow row in filteredRows)
                        {
                            // Create a new measurement for the provided field level information
                            Measurement measurement = new Measurement();

                            // Parse primary measurement identifier
                            signalID = row["SignalID"].ToNonNullString(Guid.Empty.ToString()).ConvertToType<Guid>();

                            // Set measurement key if defined
                            if (MeasurementKey.TryParse(row["ID"].ToString(), signalID, out key))
                                measurement.Key = key;

                            // Assign other attributes
                            measurement.ID = signalID;
                            measurement.TagName = row["PointTag"].ToNonNullString();
                            measurement.Multiplier = double.Parse(row["Multiplier"].ToString());
                            measurement.Adder = double.Parse(row["Adder"].ToString());

                            subscribedMeasurements.Add(measurement);
                        }

                        if (subscribedMeasurements.Count > 0)
                        {
                            // Combine subscribed output measurement with any existing output measurement and return unique set
                            if (OutputMeasurements == null)
                                OutputMeasurements = subscribedMeasurements.ToArray();
                            else
                                OutputMeasurements = subscribedMeasurements.Concat(OutputMeasurements).Distinct().ToArray();
                        }
                    }
                    catch (Exception ex)
                    {
                        // Errors here may not be catastrophic, this simply limits the auto-assignment of input measurement keys desired for subscription
                        OnProcessException(new InvalidOperationException(string.Format("Failed to define subscribed measurements: {0}", ex.Message), ex));
                    }
                }
By the time this code is reached, base.Initialize() has already been called. This means that the outputMeasurements filter expression has already been processed and a collection of output measurements has been created. Put simply, this code creating a list of subscribed measurements and concatenating it with the already existing output measurements. It's essentially augmenting its list of OutputMeasurements with those from the ActiveMeasurement table which are flagged as Subscribed.

Stephen
Developer
Apr 29, 2013 at 5:03 PM
That is exactly the key I needed for solving this puzzle!

Looks like there were several issues here causing a tremendous amount of confusion:
  1. An error in the original filter expression syntax cause no measurements to be added to the outputMeasurements during the base.Initialize() call. This is why it appear that the Subscribed flag was the only mechanism that could get measurements through the stream.
  2. Not all phasors in the DB have been redefined from their default Phase value of '+'. Either that or they were not defined automatically with the PMU connection tester and subsequenct connection and configuration files. Therefore, some but not all A, B, & C measurements were making it through the stream with a correct filter syntax. :( :( :(
I am on route to fix these problems. I will post if this fixes everything.

Thanks for all the help, Stephen!
Coordinator
Apr 29, 2013 at 5:18 PM
Sounds like you're making progress - FYI, keep in mind that IEEE C37.118-2005 has no mechanism to identify the phase of measured line (37.118-2012 does but not many people are using that yet). Basically that means if you didn't define the proper phase for the phasor measurements when you defined the device they will all default to positive sequence ('+')... You can always go back and correct this - but in your case you may need to correct it in multiple nodes...

Good luck!
Ritchie
Developer
May 2, 2013 at 1:24 PM
Just confirming that this has fixed all of my Subscription woes!

Thanks

Kevin