Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I had always used ******* but for the past 2 weeks have been trying DTN IQFeed. Customer support has been extraordinary. They call just to make sure your problem hasn't recurred." - Comment from Public Forum
"This is an excellent value, the system is generous (allowing for 500 stocks) and stable (and really is tick-by-tick), and the support is fantastic." - Comment from Shirin via Email
"Just a quick one to say I'm very impressed so far :) The documentation for developers is excellent and I've quickly managed to get an app written to do historical downloads. The system is very robust and pretty quick considering the extent of data that's available. The support guys have been very helpful too, in combination with the forums it's been plain sailing so far!" - Comment from Adam
"I just wanted to let u know that your data feed/service is by far the best!!! Your unfiltered tick data is excellent for reading order flow and none of your competitors delivers this quality of data!" - Comment from Peter via Email
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"Version 4.0.0.2 has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"There is no doubt that IQFeed is the best data provider. I am very satisfied with your services. And IQFeed is the only one that I would recommend to my friends. Now, most of them are using your product in China." - Comment from Zhezhe
"Everything is working great with the API. I love it." - Comment from Calvin
"My broker in Davenport suggested I give you a try as he uses your service and says its the best." - Comment from Bill via RT Chat
"I used to have *******, but they are way more money for the same thing. I have had no probs with data from DTN since switching over." - Comment from Public Forum Post
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »Archive (2017 and earlier) »IQFeed Developer Support »Duplicate Q Messages
Author Topic: Duplicate Q Messages (1 messages, Page 1 of 1)

jwelch
-Interested User-
Posts: 1
Joined: Mar 15, 2018


Posted: Mar 15, 2018 10:38 PM          Msg. 1 of 1
TLDR; - Is it normal to expect the occasional duplicate Q message in the feed? Or are these meant to represent two genuine ticks that somehow end up with exactly the same data (including TickID?)

Details:
After doing some analysis on the feed data I receive, there are occasional duplicate tick messages (as in they have exactly the same crc64 value)

For Example, here is a summary of the duplicate lines found in a sample feed from 3/7/18.


Writing CRC Collisions to: IQFeed (3-7-2018).crc
Writing duplicate entries to: IQFeed (3-7-2018).dup
Processed 11734028 ticks
Found 2045 duplicate lines in the file
T: 0
Q: 1983
P: 2
S: 0
F: 60
CRC: 0 - No collisions

Ignoring the F and P duplicates as those might expected... here is a sample of some Q messages which had the same CRC64 value.


Duplicate Line Found on Hash [f612fb7a7fa49e97]
Line 1 [864989]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Line 2 [864991]: Q,HPQ,7,23.8700,-0.0800,-0.003340,366424,23.9506,23.7100,23.8600,23.8700,1100,1300,,0.2406,,23.7451,23.9500,0.0100,,,N,,,,-0.0800,0.0000,10.33,-0.970115,-0.0100,-0.0100,0.1249,1,0.010080,39086313,14,4,,9878149,,,,,1897,23.8356,877019989,N,,7,11,,200,09:44:32.512085,7,23.8700,200,09:44:32.512085,01,7,23.8700,200,09:44:32.512085,7,ba,09:44:32.595686,09:44:32.595686,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [cada7e85403bc991]
Line 1 [866805]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Line 2 [866807]: Q,CTL,7,18.1000,0.0400,0.002215,501587,18.1600,17.8802,18.0900,18.1000,900,800,,0.2798,,17.9100,18.0600,0.0100,,,N,,,,0.0400,0.0000,8.19,-0.964172,-0.0100,-0.0100,0.1900,1,0.015459,19364502,14,4,,12264762,,,,,2470,18.0065,878202946,N,,7,7,,900,09:44:34.129734,19,18.1000,900,09:44:34.129734,01,19,18.1000,900,09:44:34.129734,19,ba,09:44:34.138719,09:44:34.138719,03/07/2018,03/07/2018,03/07/2018,
Duplicate Line Found on Hash [3ee933e4adea4f37]
Line 1 [869514]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,
Line 2 [869516]: Q,T,7,36.7350,-0.1350,-0.003662,1214701,36.9800,36.5600,36.7400,36.7500,1000,1300,,0.4200,,36.5900,36.8700,0.0100,,,N,,,,-0.1350,0.0000,7.70,-0.952953,0.0100,0.0100,0.1450,1,0.011433,225609215,14,4,,17462057,,,,,5300,36.7725,878205011,N,,11,5,,100,09:44:35.459813,19,36.7350,100,09:44:35.459813,01,19,36.7350,100,09:44:35.459813,19,ba,09:44:37.923864,09:44:37.923864,03/07/2018,03/07/2018,03/07/2018,


The number in square brackets e.g. 'Line 1 [#]' is a local message identifier. Clearly the messages are received very close to one another in time.

This is with the field parameters as follows:

S,CURRENT UPDATE FIELDNAMES,Symbol,Exchange ID,Last,Change,Percent Change,Total Volume,High,Low,Bid,Ask,Bid Size,Ask Size,Tick,Range,Open Interest,Open,Close,Spread,Settle,Delay,Restricted Code,Net Asset Value,Average Maturity,7 Day Yield,Extended Trading Change,Extended Trading Difference,Price-Earnings Ratio,Percent Off Average Volume,Bid Change,Ask Change,Change From Open,Market Open,Volatility,Market Capitalization,Fraction Display Code,Decimal Precision,Days to Expiration,Previous Day Volume,Open Range 1,Close Range 1,Open Range 2,Close Range 2,Number of Trades Today,VWAP,TickID,Financial Status Indicator,Settlement Date,Bid Market Center,Ask Market Center,Available Regions,Last Size,Last Time,Last Market Center,Most Recent Trade,Most Recent Trade Size,Most Recent Trade Time,Most Recent Trade Conditions,Most Recent Trade Market Center,Extended Trade,Extended Trade Size,Extended Trade Time,Extended Trade Market Center,Message Contents,Ask Time,Bid Time,Last Date,Extended Trade Date,Most Recent Trade Date


Note that TickID is included in the message, and is identical on lines which are found to have the same hash.

So, given the rather low occurrence of these duplicates (< 0.02%) this is probably not much of a concern for most analyses, but I was curious if this is expected behavior in the feed either way.

Thanks!
 

 

Time: Thu May 2, 2024 11:18 AM CFBB v1.2.0 9 ms.
© AderSoftware 2002-2003