Hello everybody !

i installed the OpenPDC 1.4 SP2 and it work well, but i have created the AdoAdapters to store data to mysql databse with the configuration like below:

Type Name: AdoAdapters.AdoOutputAdapter

Assembly Name: AdoAdapters.dll

ConnectionString:    tableName=adapt;IDFieldName=ID;TagNameFieldName=TagName;SignalIDFieldName=SignalID;SourceFieldName=Source;TimestampQualityIsGoodFieldName=TimestampQualityIsGood;ValueQualityIsGoodFieldName=ValueQualityIsGood;TimestampFieldName=Timestamp;ValueFieldName=Value;timestampFormat=yyyy-MM-dd HH:mm:ss.fff;dbconnectionString={server=host;database=database;uid=user;pwd=password''};dataProviderString={AssemblyName={MySql.Data, Version=6.2.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d}; ConnectionType=MySql.Data.MySqlClient.MySqlConnection; AdapterType=MySql.Data.MySqlClient.MySqlDataAdapter}

The schema of the table:

CREATE TABLE `adapt` (
  `ID` int(30) DEFAULT NULL,
  `SignalID` varchar(100) DEFAULT NULL,
  `TagName` varchar(200) DEFAULT NULL,
  `Source` varchar(100) DEFAULT NULL,
  `Timestamp` varchar(50) NOT NULL,
  `TimestampQualityIsGood` tinyint(1) DEFAULT NULL,
  `ValueQualityIsGood` tinyint(1) DEFAULT NULL,
  `Value` double DEFAULT NULL,
  KEY `NewIndex1` (`Timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=latin

the AdoAdapters can't save data to the Database and notice in OpenPDC Console:

[AdoAdapters] Measurement property not found: SignalID

[AdoAdapters] Measurement property not found: Source

[AdoAdapters] Measurement property not found: TimestampQualityIsGood

[AdoAdapters] Measurement property not found: ValueQualityIsGood

[AdoAdapters] Measurement property not found: Timestamp

[AdoAdapters] Measurement property not found: Value

and the value of every field in adapt table is NULL.

Please help me to solve the problem !

Thank in advance any help !

GiaLong

Could someone help me to fix the above problem, Please !

Thanks in advance !

GiaLong

Did you get this resolved?

Thanks Ritchie !

i don't know how to resolve this problem.

i think that have something was changed in the new version of AdoAdapter.

i installed the OpenPDC 1.4 SP2 to test and i get this problem.

Please help me resolve it.

Thank in advance !

GiaLong

I found the problem and released an AdoOutputAdapter Hotfix on the Downloads page. Please place the updated AdoAdapters.dll into the installation folder for the OpenPDC and let us know if that resolves the problem.

Thanks,
Stephen 

Thank you so much for your reply, Staphen !

i tried to test with the new AdoAdapters.dll that you suggested but i get the same problem like the old one, you can see in the image below:

http://www.mediafire.com/?ah8iddqdq9ewp38

i used the same configuration for the new Adapter like i posted above.

Please help me to solve this problem !

Thanks in advance for your help !

GiaLong

I believe the problem you're seeing now is due to changes in the properties of IMeasurement. I would suggest changing your table schema and connection string similar to the following.

CREATE TABLE `adapt` (
  `MeasurementKey` varchar(100) DEFAULT NULL,
  `ID` varchar(100) DEFAULT NULL,
  `TagName` varchar(200) DEFAULT NULL,
  `Timestamp` varchar(50) NOT NULL,
  `StateFlags` int(30) DEFAULT NULL,
  `Value` double DEFAULT NULL,
  KEY `NewIndex1` (`Timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=latin

tableName=adapt;KeyFieldName=MeasurementKey;IDFieldName=ID;TagNameFieldName=TagName;TimestampFieldName=Timestamp;StateFlagsFieldName=StateFlags;ValueFieldName=Value;timestampFormat=yyyy-MM-dd HH:mm:ss.fff;dbconnectionString={server=host;database=database;uid=user;pwd=password};dataProviderString={AssemblyName={MySql.Data, Version=6.2.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d}; ConnectionType=MySql.Data.MySqlClient.MySqlConnection; AdapterType=MySql.Data.MySqlClient.MySqlDataAdapter}

Additionally, I fixed various bugs in the AdoOutputAdapters related to the previous Hotfix, and have since replaced the file on the Downloads page. Please revisit that page and get the latest Hotfix.

Stephen

Thank you so much Stephen !

the new AdoAdapter work well, it can save data to Mysql Database now. but it have a little problem like the image below:

http://www.mediafire.com/?n70zx04a76zdlpo

i don't know the meaning of this problem.

could you help me about this.

i tested the new AdoAdapter with Shelby PMU simulator only and i think it work well except a little problem like above, i'll test it with my system and real PMUs in next Monday then tell you the result.

Thank you so much for your help, Stephen !

GiaLong

Please download the AdoOutputAdapter Hotfix one (hopefully) last time. The latest version should fix the flooding problem.

Thanks,
Stephen 

Thanks you so much for your help, Stephen !

i'm using the latest AdoAdapters and i have another problem:

1. the latest AdoAdapters work well(it can save data to Mysql Database with no problem) with only SHELBY PMU simulator connected.

2. the problem appear with only one real PMU connected ( i disable the SHELBY), the OpenPDC show the warning like in the image below and the AdoAdapter can't save data to Mysql Database in sometime (the AdoAdapters send data to Mysql Database only fews time after  i restart the OpenPDC service then stop to save data to the Mysql Database, i try to restart the OpenPDC many times and i get the same result.) and the OpenPDC service use 50% of the CPU with only one real PMU connected.

http://www.mediafire.com/?bnb6h7bdiafniu1

3. sometime the OpenPDC Console display another warning but not so much like the image below:

http://www.mediafire.com/i/?m6m5c55ckvldf75

could you tell me what is the meaning of this warning, Please ?

above is the situation of my system and the problem that i'm having now.

My server:

OS:  Windows 7 64 bits.

OpenPDC:  V1.4 SP2 64 bits

CPU: Corei 7 (4 cores, 8 threads).

Ram: 24GB (15GB is used for Ramdisk, remain 9GB for system running)

Please help me to solve the problem.

Thank in advance Stephen !

GiaLong

GiaLong,

"There are XXXXXX unprocessed measurements in the output queue." This warning message means one of two things. Either the adapter is receiving more measurements than it can process and is getting overrun, or the adapter has encountered an error and stopped processing measurements. I ran a few tests using multiple file based inputs and the maximum throughput on my machine using the AdoOutputAdapter to MySQL was about 1700 measurements per second. Your machine should be capable of pushing at least that much, so I think the adapter most likely encountered an error in this case.

"memory exhausted near '...' at line X" This error message appears because the bulk insert I implemented is too large. If there are too many measurements being processed at once, the adapter builds an enormous bulk insert query and the database engine refuses it. To fix this, I put up another version of the Hotfix on the Downloads page. This new version limits the size of bulk inserts so that you won't receive that "memory exhausted" error. The default limit is 1024 measurements per bulk insert, but can be configured via the connection string. For instance:

tableName=adapt;KeyFieldName=MeasurementKey;IDFieldName=ID;TagNameFieldName=TagName;TimestampFieldName=Timestamp;StateFlagsFieldName=StateFlags;ValueFieldName=Value;timestampFormat=yyyy-MM-dd HH:mm:ss.fff;dbconnectionString={server=host;database=database;uid=user;pwd=password};dataProviderString={AssemblyName={MySql.Data, Version=6.2.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d}; ConnectionType=MySql.Data.MySqlClient.MySqlConnection; AdapterType=MySql.Data.MySqlClient.MySqlDataAdapter}; bulkInsertLimit=2048

You can play around with that setting if you want. I'm hoping this update fixes both of the problems you mentioned in your post.

Thanks,
Stephen 

Thank you so much for your help, Stephen !

i tried with the latest AdoAdapter and the OpenPDC V1.4 SP2 and i configured the Historian like your suggestion with 2 case that the bulkInsertLimit = 1024 and bulkInsertLimit = 2048

Six PMUs were connected to the server and all of PMUs working at 25frames/second of Data Rate. i created 3 Historians for 6 PMUs ( it mean 1 Historian save data for 2 PMUs).

- The server work well at the begining ( about 2 or 3 hours after i restart the OpenPDC): The OpenPDC use about 10-15% of CPU, The Mysql Database use from 5 to 20 % of CPU and no warning about "Unprocessed Measurement in the output queue"

- after 2 or 3 hours working the problem appear:

+ The OpenPDC Console appear the warning for 3 Historians: "<Historian Name> There are xxxx unprocessed measurements in the ouput queue" then dump the queue when xxxx number up to 500000

+ The OpenPDC use from 35 to 40 % of CPU.

+ The Mysql use from 5 to 20 % of CPU.

So i don't know why the OpenPDC use a lot of CPU after few hours working. could you explain and help me to solve the problem ?

Please show me the best way to manage 100 PMUs with the OpenPDC and AdoAdapter to save data to Mysql Database without lost data !?

Thank in advance any help !

GiaLong

GiaLong,

As I mentioned before, the message you're receiving indicates that the system has either encountered an error and stopped processing measurements or it has become overwhelmed and cannot process the measurements fast enough. Based on your explanation this time, it sounds like the system is having trouble keeping up with the volume of measurements you are storing.

There's likely an I/O bottleneck somewhere that's causing output to work more slowly than input. When this happens, the openPDC will begin to pool the measurements, then it will attempt to process them all at once. This explains why you see an increase in CPU load.

In order to fix this problem, it's important to identify the bottleneck. The best I can do is make a guess and say that, at least performance-wise, you may benefit from distributing the data to separate MySQL instances on separate machines.

I would also like to put forth that based on my estimates, pushing data from 100 PMUs to your RAM disk would likely fill it up in less than an hour.

Stephen

Hello everybody, I was following this post and I also would have a question for Stephen:
What do you mean when you talk about measures processed?  do you mean only synchrophasors, or even measures of power, frequency, digital inputs , and others?
I read here that you've reached the limit with your test around 1700 measurements per second.

now my questions are:

1. What is the structure of your system hardware for this test? which processor are you using and which type of disk??

2. GiaLong  uses 6 PMUs that send 25frames/sec. Each PMU manage 32 measurement Channels so the number of SynchroPhasor equal 32 and the number of measurements is equal to 64 (module and phase for each channel).

So, the amount of measurements generated per second is around  9600 measurements /sec  that must be stored in  mysql db.

is this amount of data managable by openpdc mysql adapter using one icore-7 (3.2Ghz) cpu???

Anyone knows, by his own experience, which is the maximum amount of measurements using a different type of processor???

Anybody have experience with a system that produce more or less 800 synchrophasors send at 25frames/sec, and stored in db for historian purpose???

thanks in advance

Hello logiclab01,

For each of the six devices, the openPDC parses the frames and converts them into raw measurements (this does include measures of power, frequency, digital inputs, and others). The AdoOutputAdapter receives the raw measurements, builds a database query, and runs the query. When I talk about measurements being processed, I have been referring to the processing that the AdoOutputAdapter does after receiving the measurements.

I think the reason that the CPU load goes up is because the system can normally spend some time sitting idle while waiting for the next frame of measurements to arrive. When the output falls behind, it no longer has any time to spend sitting idle and has to process the measurements non-stop. Thus, I do not think this test is CPU-bound, but rather I/O-bound.

1.
OS: Windows 7 64-bit
CPU: Intel Core i5-760 (4 cores, 4 threads)
RAM: 8 GB
Disk: 1 TB, 5400 RPM

2. I did not run my test for 2 to 3 hours, but when I ran my test, I was increasing my measurement throughput by one sample file in each test. Using four sample file inputs (1680 measurements per second), I was able to archive to MySQL without getting backed up. The moment I added a fifth device (2100 measurements per second), I started seeing my data get backed up. The hardware that GiaLong is using is significantly better than mine, so I expect that he will have a higher upper limit, but I also suspect that he may have already reached that limit with 9600 measurements per second.

Increasing the bulk insert limit might help. My testing has shown that the AdoOutputAdapter performs better when there are fewer database queries to be sent to MySQL. The thing to keep in mind is that if the limit is increased too much, the errors previously mentioned in this thread will come back.

As for producing ~800 synchrophasors at 25 frames per second, the openPDC's local historian is capable of storing much more than that. If you must use a database, I imagine that there are probably better options than MySQL in terms of performance. Maybe someone else can comment.

Thanks,
Stephen

thank you so much Stephen !

my system seem work well now.

thank you so much.

could you show me a document or something else to describe the structure of the historian archive files, Please ? 

i want to develop a function to get data from the historian archive file then insert this data the database, but i don't know clearly about the structure of the historian archive files.

and another question is that:

can i save data to historian archive files by historian local adapter and save data to mysql database by AdoAdapter  for one PMU in the same time ?

Please help me !

Thank you so much !

GiaLong

Yes, we have a document that describes the structure of the historian archive files. There is a link to it on the FAQ.
http://openpdc.codeplex.com/wikipage?title=FAQ#binary_format_of_history_documented

Additionally, there is an API within the TVA.Historian library that allows you to access the historian data from within a .NET application. A little more information can be found at the bottom of this page.
http://openpdc.codeplex.com/wikipage?title=Data%20Access%20Options

As for your other question, it certainly is possible to save the data using the local historian and the AdoAdapter simultaneously. You are probably defining your adapters in the Historian table. The Historian table is designed to automatically append the sourceids parameter to the connection string, which allows it to filter down to only the measurements which define its ID as their HistorianID. If you instead define the AdoOutputAdapter as a CustomOutputAdapter (go to "Adapters > Output Adapters" in the openPDC Manager), you can then define your input measurement keys manually via the connection string. The syntax for the input measurement keys can be found on this page.
http://openpdc.codeplex.com/wikipage?title=Connection%20Strings#input_and_output_syntax

Good luck!
Stephen 

Hello Stephen !

thank you so much for your help !

i'm developing the software to get data from the historian archive files then put data to the database with C language.

i can read the Footer, the Block Map, the Data Block.

i try to get data of the first Data Block and i can get the data of the Data Points inside the first Data Block but the problem is that it appear many times one timestamp with different value of the Value column and the milliseconds of the first Data Point is not zero.

so i have some question is that:

1. why is the timestamp of the first Data Point on the first Data Block not equal with the Starting Time ?

2. why do it have duplicate timestamps that have difference value of the Value column on one Data Block ?

Please explain me and help me to solve the problem !

thank in advance !

PS:

for example i have:

the Starting Time: 2012-03-26 07!24!13.620 ( the Starting Time from the file name)

The PMUs working at 50 frames/second of Data Rate.

following is the result that i have when read the Historian Archive File with my software:

                Timestamp               ,flag    ,   value    , milliseconds

2012/03/26 07h:24m:14s:240,000D,0.000000,620 this is the timestamp of the first Data Point but it's not zero so the Timestamp is not equal the Starting Time
2012/03/26 07h:24m:14s:260,000D,0.000000,640
2012/03/26 07h:24m:14s:280,000D,0.000000,660
2012/03/26 07h:24m:14s:299,000D,0.000000,679
2012/03/26 07h:24m:14s:320,000D,0.000000,700
2012/03/26 07h:24m:14s:340,000D,0.000000,720
2012/03/26 07h:24m:14s:360,000D,-0.000000,740
2012/03/26 07h:24m:14s:380,000D,-0.000000,760
2012/03/26 07h:24m:14s:400,000D,0.000000,780
2012/03/26 07h:24m:14s:420,000D,0.000000,800
2012/03/26 07h:24m:14s:440,000D,0.000000,820
2012/03/26 07h:24m:14s:460,000D,0.000000,840
2012/03/26 07h:24m:14s:480,000D,-0.000000,860
2012/03/26 07h:24m:14s:500,000D,0.000000,880
2012/03/26 07h:24m:14s:520,000D,0.000000,900
2012/03/26 07h:24m:14s:540,000D,-0.000000,920
2012/03/26 07h:24m:14s:560,000D,-0.000000,940
2012/03/26 07h:24m:14s:580,000D,0.000000,960
2012/03/26 07h:24m:14s:600,000D,0.000000,980
2012/03/26 07h:24m:13s:620,000D,0.000000,0  //this Timestamp is equal the Starting Time but it's not the first Data Point of the Data Block.
2012/03/26 07h:24m:13s:640,000D,0.000000,20
2012/03/26 07h:24m:13s:660,000D,0.000000,40
2012/03/26 07h:24m:13s:679,000D,0.000000,59
2012/03/26 07h:24m:13s:700,000D,-0.000000,80
2012/03/26 07h:24m:13s:720,000D,0.000000,100
2012/03/26 07h:24m:13s:740,000D,0.000000,120
2012/03/26 07h:24m:13s:760,000D,-0.000000,140
2012/03/26 07h:24m:13s:780,000D,0.000000,160
2012/03/26 07h:24m:13s:799,000D,-0.000000,179
2012/03/26 07h:24m:13s:820,000D,-0.000000,200
2012/03/26 07h:24m:13s:840,000D,0.000000,220
2012/03/26 07h:24m:13s:860,000D,0.000000,240
2012/03/26 07h:24m:13s:880,000D,-0.000000,260
2012/03/26 07h:24m:13s:900,000D,0.000000,280
2012/03/26 07h:24m:13s:920,000D,-0.000000,300
2012/03/26 07h:24m:13s:940,000D,0.000000,320
2012/03/26 07h:24m:13s:960,000D,0.000000,340
2012/03/26 07h:24m:13s:980,000D,-0.000000,360
2012/03/26 07h:24m:14s:000,000D,0.000000,380
2012/03/26 07h:24m:14s:020,000D,0.000000,400
2012/03/26 07h:24m:14s:040,000D,-0.000000,420
2012/03/26 07h:24m:14s:060,000D,-0.000000,440
2012/03/26 07h:24m:14s:080,000D,-0.000000,460
2012/03/26 07h:24m:14s:100,000D,0.000000,480
2012/03/26 07h:24m:14s:120,000D,0.000000,500
2012/03/26 07h:24m:14s:140,000D,0.000000,520
2012/03/26 07h:24m:14s:160,000D,0.000000,540
2012/03/26 07h:24m:14s:179,000D,-0.000000,559
2012/03/26 07h:24m:14s:200,000D,0.000000,580
2012/03/26 07h:24m:14s:220,000D,-0.000000,600
2012/03/26 07h:24m:14s:240,000D,-0.000000,620
2012/03/26 07h:24m:14s:260,000D,0.000000,640
2012/03/26 07h:24m:14s:280,000D,0.000000,660
2012/03/26 07h:24m:14s:299,000D,-0.000000,679
2012/03/26 07h:24m:14s:320,000D,0.000000,700
2012/03/26 07h:24m:14s:340,000D,0.000000,720
2012/03/26 07h:24m:14s:360,000D,0.000000,740
2012/03/26 07h:24m:14s:380,000D,0.000000,760
2012/03/26 07h:24m:14s:400,000D,-0.000000,780
2012/03/26 07h:24m:14s:420,000D,0.000000,800
2012/03/26 07h:24m:14s:440,000D,-0.000000,820
2012/03/26 07h:24m:14s:460,000D,-0.000000,840
2012/03/26 07h:24m:14s:480,000D,0.000000,860
2012/03/26 07h:24m:14s:500,000D,0.000000,880
2012/03/26 07h:24m:14s:520,000D,0.000000,900
2012/03/26 07h:24m:14s:540,000D,-0.000000,920
2012/03/26 07h:24m:14s:560,000D,0.000000,940
2012/03/26 07h:24m:14s:580,000D,0.000000,960
2012/03/26 07h:24m:14s:600,000D,-0.000000,980
2012/03/26 07h:24m:13s:620,000D,-0.000000,0
2012/03/26 07h:24m:13s:640,000D,-0.000000,20
2012/03/26 07h:24m:13s:660,000D,-0.000000,40
2012/03/26 07h:24m:13s:679,000D,0.000000,59
2012/03/26 07h:24m:13s:700,000D,0.000000,80
2012/03/26 07h:24m:13s:720,000D,-0.000000,100
2012/03/26 07h:24m:13s:740,000D,-0.000000,120
2012/03/26 07h:24m:13s:760,000D,0.000000,140
2012/03/26 07h:24m:13s:780,000D,-0.000000,160
2012/03/26 07h:24m:13s:799,000D,-0.000000,179
2012/03/26 07h:24m:13s:820,000D,0.000000,200
2012/03/26 07h:24m:13s:840,000D,0.000000,220
2012/03/26 07h:24m:13s:860,000D,0.000000,240
2012/03/26 07h:24m:13s:880,000D,0.000000,260
2012/03/26 07h:24m:13s:900,000D,-0.000000,280
2012/03/26 07h:24m:13s:920,000D,0.000000,300
2012/03/26 07h:24m:13s:940,000D,-0.000000,320
2012/03/26 07h:24m:13s:960,000D,0.000000,340
2012/03/26 07h:24m:13s:980,000D,0.000000,360
2012/03/26 07h:24m:14s:000,000D,0.000000,380
2012/03/26 07h:24m:14s:020,000D,-0.000000,400
2012/03/26 07h:24m:14s:040,000D,0.000000,420
2012/03/26 07h:24m:14s:060,000D,-0.000000,440
2012/03/26 07h:24m:14s:080,000D,-0.000000,460
2012/03/26 07h:24m:14s:100,000D,0.000000,480
2012/03/26 07h:24m:14s:120,000D,-0.000000,500
2012/03/26 07h:24m:14s:140,000D,-0.000000,520
2012/03/26 07h:24m:14s:160,000D,-0.000000,540
2012/03/26 07h:24m:14s:179,000D,-0.000000,559
2012/03/26 07h:24m:14s:200,000D,0.000000,580
2012/03/26 07h:24m:14s:220,000D,-0.000000,600
2012/03/26 07h:24m:14s:240,000D,0.000000,620
2012/03/26 07h:24m:14s:260,000D,0.000000,640
2012/03/26 07h:24m:14s:280,000D,0.000000,660
2012/03/26 07h:24m:14s:299,000D,0.000000,679
2012/03/26 07h:24m:14s:320,000D,-0.000000,700
2012/03/26 07h:24m:14s:340,000D,-0.000000,720
2012/03/26 07h:24m:14s:360,000D,0.000000,740
2012/03/26 07h:24m:14s:380,000D,0.000000,760
2012/03/26 07h:24m:14s:400,000D,0.000000,780
2012/03/26 07h:24m:14s:420,000D,-0.000000,800
2012/03/26 07h:24m:14s:440,000D,-0.000000,820
2012/03/26 07h:24m:14s:460,000D,0.000000,840
2012/03/26 07h:24m:14s:480,000D,-0.000000,860
2012/03/26 07h:24m:14s:500,000D,0.000000,880
2012/03/26 07h:24m:14s:520,000D,-0.000000,900
2012/03/26 07h:24m:14s:540,000D,-0.000000,920
2012/03/26 07h:24m:14s:560,000D,0.000000,940
2012/03/26 07h:24m:14s:580,000D,-0.000000,960
2012/03/26 07h:24m:14s:600,000D,-0.000000,980
2012/03/26 07h:24m:13s:620,000D,0.000000,0
2012/03/26 07h:24m:13s:640,000D,-0.000000,20
2012/03/26 07h:24m:13s:660,000D,-0.000000,40
2012/03/26 07h:24m:13s:679,000D,-0.000000,59
2012/03/26 07h:24m:13s:700,000D,0.000000,80
2012/03/26 07h:24m:13s:720,000D,0.000000,100
2012/03/26 07h:24m:13s:740,000D,0.000000,120
2012/03/26 07h:24m:13s:760,000D,0.000000,140
2012/03/26 07h:24m:13s:780,000D,0.000000,160
2012/03/26 07h:24m:13s:799,000D,-0.000000,179
2012/03/26 07h:24m:13s:820,000D,0.000000,200
2012/03/26 07h:24m:13s:840,000D,0.000000,220
2012/03/26 07h:24m:13s:860,000D,-0.000000,240
2012/03/26 07h:24m:13s:880,000D,-0.000000,260
2012/03/26 07h:24m:13s:900,000D,-0.000000,280
2012/03/26 07h:24m:13s:920,000D,0.000000,300
2012/03/26 07h:24m:13s:940,000D,-0.000000,320
2012/03/26 07h:24m:13s:960,000D,0.000000,340
2012/03/26 07h:24m:13s:980,000D,0.000000,360
2012/03/26 07h:24m:14s:000,000D,0.000000,380
2012/03/26 07h:24m:14s:020,000D,-0.000000,400
2012/03/26 07h:24m:14s:040,000D,-0.000000,420
2012/03/26 07h:24m:14s:060,000D,-0.000000,440
2012/03/26 07h:24m:14s:080,000D,-0.000000,460
2012/03/26 07h:24m:14s:100,000D,-0.000000,480
2012/03/26 07h:24m:14s:120,000D,0.000000,500
2012/03/26 07h:24m:14s:140,000D,-0.000000,520
2012/03/26 07h:24m:14s:160,000D,-0.000000,540
2012/03/26 07h:24m:14s:179,000D,-0.000000,559
2012/03/26 07h:24m:14s:200,000D,-0.000000,580
2012/03/26 07h:24m:14s:220,000D,-0.000000,600
2012/03/26 07h:24m:14s:240,000D,0.000000,620
2012/03/26 07h:24m:14s:260,000D,-0.000000,640
2012/03/26 07h:24m:14s:280,000D,-0.000000,660
2012/03/26 07h:24m:14s:299,000D,-0.000000,679
2012/03/26 07h:24m:14s:320,000D,-0.000000,700
2012/03/26 07h:24m:14s:340,000D,-0.000000,720
2012/03/26 07h:24m:14s:360,000D,-0.000000,740
2012/03/26 07h:24m:14s:380,000D,-0.000000,760
2012/03/26 07h:24m:14s:400,000D,0.000000,780
2012/03/26 07h:24m:14s:420,000D,0.000000,800
2012/03/26 07h:24m:14s:440,000D,-0.000000,820
2012/03/26 07h:24m:14s:460,000D,0.000000,840
2012/03/26 07h:24m:14s:480,000D,-0.000000,860
2012/03/26 07h:24m:14s:500,000D,0.000000,880
2012/03/26 07h:24m:14s:520,000D,0.000000,900
2012/03/26 07h:24m:14s:540,000D,-0.000000,920
2012/03/26 07h:24m:14s:560,000D,0.000000,940
2012/03/26 07h:24m:14s:580,000D,-0.000000,960
2012/03/26 07h:24m:14s:600,000D,0.000000,980
2012/03/26 07h:24m:13s:620,000D,-0.000000,0
2012/03/26 07h:24m:13s:640,000D,0.000000,20
2012/03/26 07h:24m:13s:660,000D,0.000000,40
2012/03/26 07h:24m:13s:679,000D,0.000000,59
2012/03/26 07h:24m:13s:700,000D,-0.000000,80
2012/03/26 07h:24m:13s:720,000D,-0.000000,100
2012/03/26 07h:24m:13s:740,000D,0.000000,120
2012/03/26 07h:24m:13s:760,000D,-0.000000,140
2012/03/26 07h:24m:13s:780,000D,0.000000,160
2012/03/26 07h:24m:13s:799,000D,0.000000,179
2012/03/26 07h:24m:13s:820,000D,-0.000000,200
2012/03/26 07h:24m:13s:840,000D,0.000000,220
2012/03/26 07h:24m:13s:860,000D,0.000000,240
2012/03/26 07h:24m:13s:880,000D,-0.000000,260
2012/03/26 07h:24m:13s:900,000D,-0.000000,280
2012/03/26 07h:24m:13s:920,000D,0.000000,300
2012/03/26 07h:24m:13s:940,000D,0.000000,320
2012/03/26 07h:24m:13s:960,000D,-0.000000,340
2012/03/26 07h:24m:13s:980,000D,-0.000000,360
2012/03/26 07h:24m:14s:000,000D,-0.000000,380
2012/03/26 07h:24m:14s:020,000D,0.000000,400
2012/03/26 07h:24m:14s:040,000D,-0.000000,420
2012/03/26 07h:24m:14s:060,000D,0.000000,440
2012/03/26 07h:24m:14s:080,000D,-0.000000,460
2012/03/26 07h:24m:14s:100,000D,-0.000000,480
2012/03/26 07h:24m:14s:120,000D,-0.000000,500
2012/03/26 07h:24m:14s:140,000D,0.000000,520
2012/03/26 07h:24m:14s:160,000D,0.000000,540
2012/03/26 07h:24m:14s:179,000D,0.000000,559
2012/03/26 07h:24m:14s:200,000D,0.000000,580
2012/03/26 07h:24m:14s:220,000D,0.000000,600
2012/03/26 07h:24m:14s:240,000D,0.000000,620
2012/03/26 07h:24m:14s:260,000D,0.000000,640
2012/03/26 07h:24m:14s:280,000D,-0.000000,660
2012/03/26 07h:24m:14s:299,000D,-0.000000,679
2012/03/26 07h:24m:14s:320,000D,-0.000000,700
2012/03/26 07h:24m:14s:340,000D,0.000000,720
2012/03/26 07h:24m:14s:360,000D,0.000000,740
2012/03/26 07h:24m:14s:380,000D,0.000000,760
2012/03/26 07h:24m:14s:400,000D,-0.000000,780
2012/03/26 07h:24m:14s:420,000D,-0.000000,800
2012/03/26 07h:24m:14s:440,000D,0.000000,820
2012/03/26 07h:24m:14s:460,000D,0.000000,840
2012/03/26 07h:24m:14s:480,000D,0.000000,860
2012/03/26 07h:24m:14s:500,000D,0.000000,880
2012/03/26 07h:24m:14s:520,000D,0.000000,900
2012/03/26 07h:24m:14s:540,000D,0.000000,920
2012/03/26 07h:24m:14s:560,000D,0.000000,940
2012/03/26 07h:24m:14s:580,000D,0.000000,960
2012/03/26 07h:24m:14s:600,000D,-0.000000,980
2012/03/26 07h:24m:13s:620,000D,0.000000,0
2012/03/26 07h:24m:13s:640,000D,0.000000,20
2012/03/26 07h:24m:13s:660,000D,0.000000,40
2012/03/26 07h:24m:13s:679,000D,-0.000000,59

Is this data that's just in the file or is mapped through the header?

Do you get the same results if you use the .NET API?

Thanks!
Ritchie

Thank you for your reply, Ritchie !

above is data in the file( the data of all Data Point inside the first Data Block).

i can get the value of the fields: Flags( value of flag, milliseconds[from 0 to 999]), Value but the value of the OffsetTimeStamp field is alway zero so i have some question:

what is the value of the OffsetTimeStamp in the Data Point ?

How can i calculate the actual time of one Data Point ?

Please help me !

Thank in advance.

GiaLong

The timestamp for a start time for a datablock value (the base offset which starts at the 5th byte beyond the 4-byte block ID) is a little-endian encoded double precision flaoting point number (8-bytes) that represents the number of seconds (including fractional part for milliseconds).

This just represents the "starting point" for a block as a storage reference - the actual data point values contain the actual time stamp.

The data point structure is defined as follows:

        // ******************************************************************
        // *                    Binary Structure                            *
        // ******************************************************************
        // * # Of Bytes Byte Index Data Type  Property Name                 *
        // * ---------- ---------- ---------- ------------------------------*
        // * 4          0-3        Int32      Time                          *
        // * 2          4-5        Int16      Flags (Quality & Milliseconds)*
        // * 4          6-9        Single     Value                         *
        // ******************************************************************

Parsing would be similar to the following:

Flags = EndianOrder.LittleEndian.ToInt16(buffer, startIndex + 4);
Value = EndianOrder.LittleEndian.ToSingle(buffer, startIndex + 6);
Time = new TimeTag(EndianOrder.LittleEndian.ToInt32(buffer, startIndex) + ((double)(m_flags.GetMaskedValue(MillisecondMask) >> 5) / 1000));

Where millisecond mask is:

MillisecondMask = 
	Bits.Bit05 | Bits.Bit06 | Bits.Bit07 | Bits.Bit08 | Bits.Bit09 | Bits.Bit10 | Bits.Bit11 | 
	Bits.Bit12 | Bits.Bit13 | Bits.Bit14 | Bits.Bit15;

Does that help?

Thanks!
Ritchie