I notice that if I leave IQFeed connected for more than one day, often it seems to silently stop working just before the second day starts (usually about 8:30 EST). The only indication that anything is wrong seems to be that I would stop receiving time-stamp messages.

Questions:
1. Is IQFeed designed to run for more than one day?
2. Is there some kind of server reset happening at your end during this time?
3. Is the lack of time-stamp messages significant of something having gone wrong?
4. Shouldn't I be getting some sort of server disconnect message instead of 3?

>1. Is IQFeed designed to run for more than one day?
Yes.

>2. Is there some kind of server reset happening at your end during this time?
This should be the start of the market. You shouldn't have noticed anything other than a lot more data coming in.

>Is the lack of time-stamp messages significant of something having gone wrong?
Yes. You should be expecting the next time stamp on time, unless you are backing up data because you have a high CPU load and are not processing it quickly enough which would then cause it to be delayed.

>4. Shouldn't I be getting some sort of server disconnect message instead of 3?
That is odd. You should see a S,SERVER DISCONNECTED message, if IQFeed becomes disconnected from its servers. You could then issue a S,CONNECT command and the symbols you were watching should automatically be rewatched for you, when the connection is reestablished. Did you loose your connection to IQFeed?

Can you describe a little more what you are doing, so I can better help you?

---

Visit our new online forums! DTN employees actively monitor the forums to provide you the highest level of support in the industry. Our forums also has an announcements area where you can learn about new features or changes within DTN. You can even use the forums to exchange trading ideas or tips and tricks on how you use your DTN data to be more profitable.

Go to http://forums.dtniq.com now to start sharing information with fellow DTN subscribers!

I'm running a server program which is connecting to IQFeed via TCP, basically I usually stop and start this thing each day, however yesterday I left it running overnight. I have a timer which is latched on the time-stamp message as I have seen the time-stamp messages stop before. Basically everything is chugging along just fine, IQFeed is not chewing CPU or memory, in fact there is not much load at all at 8:30 EST. Then for no reason the time-stamp messages stop (I wait for 5 minutes before timing out), as do the level 1 messages. My program breaks the TCP connection to IQFeed and tries to re-connect, after a few attempts I usually get reconnected.

I log out all data from the TCP connection, and I don't see any SERVER DISCONNECTED message.

Please let me know parts you require more info. on.

Edit....
Couple of extra notes:
1. If I stop and start IQFeed from the context menu, I see the SERVER DISCONNECTED messages in my logging, so I'm definitely handling them.
2. What is the correct response for a client that receives a SERVER DISCONNECTED? (drop connection, do nothing etc).
Edited by Craig on Aug 3, 2010 at 06:40 PM
Edited by Craig on Aug 3, 2010 at 06:40 PM

This happened again today, here is my log.

07:21:00.211227 0x954c400 Tokenizer::Parse, 'T' [T,20100805 07:21:00]
07:21:03.006119 0x954c400 Tokenizer::Parse, 'F' [F,MBT,D,14.7,1918000,23.5920,16.3160,23.5920,17.2000,,,,,05/06/2009,1,7,,3242378,,1.52,6.42,7.07,12,,MOBILE TELESYSTEMS ADR,MBT MBT1,21.,1.63,,4395.0,4257.9,03/31/2010,6482.2,958435,517210,0.40 05/03/2010,0.25 01/03/2005,,t,14,4,4899,30.55,1,7,03/17/2010,08/17/2009,03/17/2010,02/05/2010,19.56,,,,,517210]
07:21:04.985537 0x954c400 Tokenizer::Parse, 'F' [F,WTW,D,12.3,277000,30.4100,24.3900,30.4100,24.3900,2.50,0.18,0.70,07/16/2010,06/30/2010,1,7,,2816451,,2.26,2.44,6.77,12,,WEIGHT WATCHERS INTL,WTW,44.,1.08,,200.0,536.1,03/31/2010,1238.0,77083,812191, , ,,p,14,4,7299,18.14,1,7,02/23/2010,03/04/2010,02/23/2010,03/04/2010,29.16,,,,,812191]
07:21:05.425600 0x954c400 Tokenizer::Parse, 'F' [F,COG,D,27.3,1852000,46.4600,28.8900,46.4600,28.8900,0.40,0.03,0.12,08/19/2010,08/03/2010,1,7,,8632381,,1.20,1.14,-1.99,12,,CABOT OIL & GAS A,COG,,1.08,,281.5,308.7,06/30/2010,805.0,104135,211111,0.50 04/02/2007,0.67 04/01/2005,,p,14,4,1311,36.82,1,7,01/06/2010,05/25/2010,01/06/2010,05/25/2010,43.59,,,,,211111]
07:21:06.305470 0x954c400 Tokenizer::Parse, 'F' [F,LTD,D,15.2,4830000,28.7800,12.5600,28.7800,18.3400,2.30,1.15,0.60,06/15/2010,06/02/2010,1,7,,9620683,,1.71,1.78,-5.59,01,,LIMITED BRANDS,LTD,84.,1.78,,3250.0,1322.0,04/30/2010,2723.0,325191,448120,0.50 05/31/2000,0.50 06/20/1990,,t,14,4,5621,38.56,1,7,04/26/2010,08/05/2009,04/26/2010,01/06/2010,19.24,,,,,448120]
07:21:07.133532 0x954c400 Tokenizer::Parse, 'F' [F,MAT,F,12.5,5870000,24.6000,17.0800,24.6000,19.0700,3.40,,0.75,12/17/2009,11/25/2009,1,1,,9689657,,1.74,1.79,5.83,12,,MATTEL,MAT,97.,0.99,,2554.6,1060.9,06/30/2010,700.0,359724,339932,0.80 03/04/1996,0.80 01/23/1995,,t,14,4,3944,39.50,1,1,04/16/2010,08/06/2009,04/16/2010,01/29/2010,19.98,,,,,339932]
07:21:10.159038 0x954c400 Tokenizer::Parse, 'F' [F,RYN,D,14.8,710000,51.2900,37.8800,51.2900,39.7000,3.90,0.50,2.00,09/30/2010,09/08/2010,1,7,,2033954,,3.42,2.15,13.89,12,,RAYONIER,RYN,87.,1.19,,509.7,175.1,06/30/2010,695.0,80255,531190,0.67 10/18/2005,0.67 06/13/2003,,t,14,4,6798,27.67,1,7,08/04/2010,11/02/2009,08/04/2010,02/10/2010,42.16,,,,,531190]
07:21:11.585490 0x954c400 Tokenizer::Parse, 'F' [F,NU,D,16.4,1337000,29.0300,22.2000,29.0300,24.6800,3.60,0.26,1.03,09/30/2010,08/30/2010,1,7,,2711446,,1.75,1.94,,12,,NORTHEAST UTILITIES,NU,81.,0.55,,1267.9,979.4,06/30/2010,4492.9,175996,221122, , ,,t,14,4,4931,20.30,1,7,08/04/2010,11/02/2009,08/04/2010,02/10/2010,25.79,,,,,221122]
07:21:12.025526 0x954c400 Tokenizer::Parse, 'F' [F,WAT,D,18.8,880000,73.7800,49.3400,73.7800,55.9700,,,,,11/26/1996,1,7,,1868824,,3.56,3.96,15.3,12,,WATERS CORP,WAT,,1.00,,1172.4,394.6,06/30/2010,500.0,92945,334516,0.50 08/28/2000,0.50 06/11/1999,,t,14,4,3826,33.13,1,7,04/28/2010,09/02/2009,04/28/2010,02/05/2010,61.96,,,,,334516]
07:21:22.642119 0x954c400 Tokenizer::Parse, 'F' [F,MAN,D,973.8,781000,63.0000,39.9400,62.4800,39.9400,1.50,0.37,0.74,06/15/2010,06/01/2010,1,7,,2731536,,0.05,1.41,,12,,MANPOWER,MAN,,1.51,,4332.4,2603.0,06/30/2010,715.6,81875,561330, , ,,t,14,4,7363,32.31,1,7,10/20/2009,06/08/2010,04/22/2010,06/08/2010,54.58,,,,,561330]
07:21:28.085517 0x954c400 Tokenizer::Parse, 'F' [F,AYE,D,11.5,1953000,27.7000,18.9650,23.9900,18.9650,2.60,0.15,0.60,09/27/2010,09/09/2010,1,7,,7388516,,2.04,2.09,,12,,ALLEGHENY ENERGY,AYE,92.,0.89,,1345.3,985.6,03/31/2010,4417.0,169579,221111,0.50 11/19/1993, ,,p,14,4,4911,24.04,1,7,09/22/2009,05/25/2010,03/23/2010,05/25/2010,23.48,,,,,221111]
07:21:44.257416 0x954c400 Tokenizer::Parse, 'F' [F,NIHD,F,22.4,1744000,43.9500,22.1100,43.9500,30.0000,,,,,,1,1,,2534322,,1.77,1.88,14.48,12,,NII HOLDINGS,NIHD,,2.11,,3791.1,1529.7,06/30/2010,3016.2,167287,517210,0.50 11/22/2005,0.33 03/23/2004,,t,14,4,4812,45.44,1,1,04/05/2010,08/27/2009,04/05/2010,02/08/2010,33.58,,,,,517210]
07:26:00.211857 0x954c400 IqFeedConnection::IqFeedConnection::ConnectionStateLive::OnTimer, no timestamp for 5 minutes

Last time-stamp is rx'd at 07:21:00.21122, I timeout at 07:26:00.211857.
As you can see, I'm receiving other messages after the last time-stamp.
Edited by Craig on Aug 5, 2010 at 02:58 PM

Thanks for the log. I do see that you received other messages after the last timestamp but it doesn't appear as though you recieved any messages after the point when the next timestamp would/should have been received. I only point this out because it is not means that we are not necessarily dealing with a Timestamp issue itself but rather an issue causing the feed to stop, yet not register a disconnection from the servers.

The main question I have at this point would be what the stats show in the connection manager at the time that this happens (or the S,STATS message that is returned on the admin port or L1 port). I suspect that since you aren't getting a disconnection message (and the feed isn't going into a reconnection attempt loop) that the stats will show you are still connected and that the seconds since last update will be increasing. If that is the case, it sounds like the actual socket connection is still connected and your machine is able to send data to the servers but the servers are unable to send anything back. Unfortunately, the only thing I've seen cause that scenario in the past is a saturated internet connection but that doesn't seem to be the case here since it is happening during low market volume times.

It did not happen today, but I have added more logging and will monitor over the coming week.
I have seen similar problems to the one you describe on TCP connections in other contexts, it depends on the keep-alive settings on the intervening hardware as to how the connection behaves. If this is the case, monitoring the time-stamp messages as I'm doing now, maybe the only way of detecting the problem.
Edited by Craig on Aug 6, 2010 at 02:23 PM

I believe I have found the problem here, hopefully this post will stop somebody else making the same mistake. I was waiting until my rx buffer was terminated with a '\n' before I tokenized the buffer. This lead to a problem that if I ended up without a '\n' at the end due to TCP fragmentation the buffer may not get processed in a timely manner. This lead to 'bursting' behavior in the CPU and stuck messages, everything seems much smoother now this is fixed.

Alas, the problem is still happening, will keep trying to track.