Hi

My datafeed has produced some strange ticks, and I was wondering if anyone can verify whether this is an IQFeed problem or whether this might be on my side.

First, I got 150+ of ticks very similar to this one:

"Q,FCNCA,F,191.1500,0.,,0,,0,0,190.1000,193.8100,100,100,,173,0.,,,0,191.1500,3.
71,,,,t,N,,,,10/13/2006,,,,,,,,16.9,,0.1,0.,,,,1994459.1,14,4,,4437,BSE-CSE-PSE-
NMS,,,,,0,,,0,,N,"

... where the time field is empty.

Then I got a whole bunch of ticks that had strange characters inserted (including commas, which caused the trouble) for example:

                                                                      v__v 
Q,EMR,D,45.8300,-40.16,-0.46703105,221700,100,86.27,85.50,86.18,86.19,U,F|00,10

            v_v
Q,SNDK,F,0.0KEY,-0.87,-0.014297453,4393187,200,61.3500,59.8700,37.46,37.48,700,

        v___v
Q,LTD,D,AA.PL,-0.26,-0.009021513,761200,200,28.82,28.49,74.0700,74.0800,2000,20

                                                                  v__v
Q,JOYG,F,24.1700,-17.7,-0.422737043,774603,100,41.8900,40.7500,40.U,F|,40.8600,

                                                                         v____v
Q,COF,D,80.24,-1.32,-0.016184404,622600,1500,81.40,80.02,80.23,80.25,200,U,D|00

       v___v
Q,MS,D,EB.AY,-1.41,-0.018356985,1616600,100,76.75,75.32,28.7000,28.7100,7100,29

       vv  v__v
Q,SBAC,F,0.INTC,-0.37,-0.01356305,388285,100,27.1100,26.5400,20.8800,20.8900,43

       v___v
Q,MS,D,EB.AY,-1.41,-0.018356985,1616600,100,76.75,75.32,75.40,75.41,200,200,,,1

      vv
Q,USB,U,D,33.29,-0.3,-0.00893123,3236000,100,33.51,33.00,33.28,33.29,6200,33700

       vv  v__v
Q,SBAC,F,0.INTC,-0.37,-0.01356305,388285,100,27.1100,26.5400,26.8800,26.9300,30

        v__v  
Q,ALEX,DU,F,45.4900,-0.53,-0.011516732,41842,100,45.9300,45.4700,45.4600,45.500

                                                                    v__v
Q,SBL,D,35.78,20.59,1.355497038,459000,500,15.19,15.05,15.06,15.07,2U,D|00,2000

       vv  v__v
Q,SBUX,F,0.SIRI,-0.56,-0.014583333,1853336,200,38.4000,37.7900,3.8300,3.8300,14

                                                                    v___v 
Q,GRP,D,49.0000,10.89,0.285751771,1209188,188,38.09,37.15,37.42,3745U,.F|,700,1
 

I only included the first part of each tick, and I indicated with "v__v" where the probable error lies. These are only a few. I got LOTS!

Please let me know if I should go back to my drawing board, or if this might be a known issue.

Many thanks
Ilan

Ilan, can you give me some more information about your setup?

What are your Computer's specs?
What is the type(and speed) of your internet connection?
What type of data are you requesting? How many symbols are you watching? Are you just watching Level1 data? etc.

Hi Steve

I am using a Dual Pentium Xeon processor with 2GB RAM and a T1 internet connection.
I am requesting level 1 data on the 1000 largest symbols in the US (Nasdaq Amex Nyse), together with 14 Forex symbols (Barclays).

I use 1 thread for the Forex and 10 threads for the equities data (100 symbols per thread). Each thread makes its own connection to the data feed.

Should I investigate whether it is one or more threads that receive the bad ticks?

Thanks for your help
Ilan

I have no idea if this is your problem, but I have seen similiar behavior if the buffers on my program are overflowing (i.e. they are receiving data from IQFeed faster than my program can process it, eventually the input stream buffer overflows and you get lines of data compressed together like that)

This generally happens if I start running a very compute-intensive program alongside my trading system or if I move my trading system to my slooow laptop.

Maybe with the "dow 12,000" stuff, you were getting more ticks than you could handle?

The reasoning I was asking about your specs is that, in the past, this issue has been primarily due to a bad connection to the server or simply not having enough bandwidth to handle the amount of data you are requesting.

In addition to FullyArticulate's suggestions, I would recommend running some line tests to verify that your connection speed hasn't degraded as well.

There is a fairly good set of testing tools located at www.broadbandreports.com/tools

It would be interesting to find out if only some of your threads are turning up the bad data or if it is all of them.
Edited by DTN_Steve_S on Oct 18, 2006 at 11:09 AM

Uhm... I left the logging on, and was alerted to the fact last night that my hard drive was full.
I think this may have been what was causing the problem...

Steve, maybe something for your troubleshooting files - "Check whether you left your logging on"

Normally I don't think it would cause a problem, but maybe when 11 threads, 1000+ symbols are writing to the same log file, which have grown larger than 30GB... it's too much.

Thanks for your help, also thanks to you, FullyArticulate
Ilan

I experience the same problem. For some more information, please refer to:

http://forums.iqfeed.net/index.cfm?page=topic&topicID=1261

P.S.
I have not yet been able to exclude the possibility of this being caused by the Internet connection.