Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)

"I noticed that ******* quotes locked up shortly after the interest rate announcement yesterday while yours stayed stable." - Comment from Ron in Utah
"I am very happy I changed. I love the product, but more so I am thrilled with Tech Support. You are knowledgeable, polite, pleasant and professional." - Comment from Pat
"The service is great, I see a noticeable improvement in my volume profiles over [broker]'s data feed" - Comment from Larry
"After all the anxiety I had with my previous data provider it is a relief not to have to worry about data speed and integrity." - Comment from Eamonn
"I am enjoying the feed very much - so superior to the broker provided feed I was previously using." - Comment from George
"I was with ******* for 4 years at $230 a month, this is a huge savings for me, GOD BLESS YOU PEOPLE," - Comment from T.S. via Email
"Awesome response, as usual. It is a sincere and refreshing pleasure to do business with DTN, compared to your competition." - Comment from Ryan
"Version has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"I used to have *******, but they are way more money for the same thing. I have had no probs with data from DTN since switching over." - Comment from Public Forum Post
"You have an excellent feed. Very few spikes for Spot Forex." - Comment from Public Forum Post
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Invalid Update Messages using IQFeed socket API
Author Topic: Invalid Update Messages using IQFeed socket API (4 messages, Page 1 of 1)

-Interested User-
Posts: 12
Joined: Oct 7, 2004

Posted: Oct 7, 2004 10:40 AM          Msg. 1 of 4
I've been receiving invalid update messages through the socket API;
first I thought it was a bug in my update-parsing process so I modified the console_stream example to duplicate the problem -- sent the program

- it subscribes to the Nasdaq100 stocks,
- reads 1000 updates,
- unsubscribes for the symbols,
- after suspending for a minute starts the iteration again.
/This modified console_stream uses CString so use MFC static/shared library + replace #include <windows.h> to #include <afs.h> in stdafx.h/

After some iterations I receive updates like:

Note the 9th column: ".PSFT". Similar happens in other price fields as well (bid, ask, last, etc).

Could somebody also compile/run the program and see what happens (pm me , I can send the code)? Ususally the longer it runs, the more invalid updates it receives.

I appreciate any ideas, comments!!!

Edited by DTN_Steve_S on Sep 19, 2011 at 09:22 AM

-VP, Product Operations-
Posts: 1746
Joined: May 3, 2004


Posted: Oct 7, 2004 10:20 PM          Msg. 2 of 4
I think you received a response via email, but for the benefit of others (since we have seen this before), this problem of corrupt data is nearly always a result of having to small a TCP buffer. Our feed is one of the fastest and most comprehensive, so while you may have been able to get away with a small TCP buffer on feeds that consolidate or filter their feed, with ours you need to open it up. I think we recommend a 64k buffer. If I am wrong, I am sure someone will correct me with the proper size they use to keep from corrupting messages.

Jay Froscheiser
DTN Market Access, LLC.

-Interested User-
Posts: 12
Joined: Oct 7, 2004

Posted: Oct 8, 2004 10:30 AM          Msg. 3 of 4
Thank you for the fast reply, I really appreciate it!

I've tried the 64k TCP buffer modification and double-checked the partial
message handling -- I could not find any bugs in it, so I would assume that
the invalid updates were not due to partial messages as they are handled by
the test application. But just in case I've started the 64k TCP buffered
application today shortly after market open and got invalid updates again
(lines from the test application's output):

Starting iteration #3
Parse error in column 3 in line:
Parse error in column 3 in line:
Parse error in column 3 in line:


Parse error in column 9 in line:
Parse error in column 9 in line:


I would be very happy if you could compile and run this test application (sent in email) on
some of your development machines, to see whether this error can be
duplicated in your environment.

Best Regards,

-Interested User-
Posts: 54
Joined: Jul 21, 2004

Posted: Oct 8, 2004 02:15 PM          Msg. 4 of 4
I too have just noticed this problem and can replicate it using small buffers and large buffers.

I don't think it has anything to do with the buffer size as long as there are no bugs in your buffer handling and you are processing new data quickly enough.

It seems to be a problem of sending too many Watch request messages too quickly. If I send max 50 Watch symbol messages at a time then it doesn't happen. I haven't bothered to fine tune it and find the magic number because it will vary on different machines. I know 100 doesn't work.

Like Lajos I have received similar message corruption but with all message types - P, F and Q (Summary, Fundamental and Update). Most of the fields are returned but a few of the fields are rubbish, e.g. Last price = ".AMAT", etc...

I have used between 256 byte and 4096 byte buffers for each read and then multiple re-reads if more data is present. Whether you handle it quickly, very quickly or extremely quickly it doesn't matter. I have only been able to test reading up to 300KB per second from the socket in one hit if data is available, and don't seem to have any problems with my buffer handling.

Corruption only happens when:
1. Sending multiple Watch requests of 100 symbols, and only
2. If IQ Connection Manager is NOT already subscribed to them

I am dispatching a total of 95 Watch request messages one at a time, consecutively and asynchronously.

I don't think this is symbol related but I'm sure if you pick the busiest Nasdaq 100 symbols you have a greater chance of seeing corruption. I had 90 Nasdaq 100 and 5 Dow symbols.

I am using DTN.IQ and when it starts I can open up the IQ Connection Manager and check that there are no symbols being watched.

So the process is:
1. Close IQConnect process/es
2. Connect to TCPIP Level 1 stream
3. Send 100 Watch request messages
4. One of the first response messages (P, F or Q) of the 80-90th symbol will have bad fields
5. Maybe a few more corrupt messages come back
6. The rest of the messages from then on are OK
7. Un-Watch all these symbols and repeat from Step 2 and you will have the same problem

However, if you just kill the app without unwatching, IQConnection Manager shows there are still 100 symbols being watched. If you repeat from Step 2 you won't have a problem - at least I haven't seen it.

Alternatively you can Send only 50 Watch request messages at a time and not have this problem.

I'm using my own C# code.

A sample bad message I detected from error logs converting string fields to numeric types - I didn't check if the fields make sense at this point - just an attempt to convert to types.

=> Disney has a Last price of ".2485", High price of "U", Low price of "F|" and Bid Size of "55.95", Market Open of "0.001"

The next message for Disney is OK.

Repeat and I get the problem with the first message for the same symbol Disney, similar and/or different fields. I could also get one more Update message for Disney that is also corrupt.

Edited by sasha on Oct 8, 2004 at 02:17 PM


Time: Mon May 16, 2022 1:14 AM CFBB v1.2.0 15 ms.
© AderSoftware 2002-2003