Join the 80,000 other DTN customers who enjoy the fastest, most reliable data available. There is no better value than DTN!

(Move your cursor to this area to pause scrolling)




"I've never had DTN go out on me since switching. ******* would go down a couple times every month when I was using them." - Comment from Bryce in AL.
"Version 4.0.0.2 has been working well for me and I appreciate that it is now a much tighter client to work with. I feel I can go to press with my own application and rely on a stable platform" - Comment from David in IA.
"IQ feed is brilliant. The support is mind-bending. What service!" - Comment from Public Forum Post
"With HUGE volume on AAPL and RIMM for 2 days, everyone in a trading room was whining about freezes, crashes and lag with *******, RealTick, TS and Cyber. InvestorRT with IQFeed was rock solid. I mean SOLID!" - Comment from Public IRC Chat
"I "bracket trade" all major news releases and I have not found one lag or glitch with DTN.IQ feed. I am very comfortable with their feed under all typical news conditions (Fed releases, employment numbers, etc)." - Comment from Public Forum
"DTN feed was the only feed that consistently matched Bloomberg feed for BID/ASK data verification work these past years......DTN feed is a must for my supply & demand based trading using Cumulative Delta" - Comment from Public Forum Post
"And by the way, have to say this. I love the IQFeed software. It's rock solid and it has a really nice API." - Comment from Thomas via RT Chat
"This beats the pants off CQG, I am definitely switching to the ProphetX 3.0!" - Comment from Stephen
"I was with ******* for 4 years at $230 a month, this is a huge savings for me, GOD BLESS YOU PEOPLE," - Comment from T.S. via Email
"If someone needs the best quality data and backfill beyond what their broker provides at a rate that is the best in the industry, I highly recommend IQFeed." - Comment from Josh via Public Forum
Home  Search  Register  Login  Recent Posts

Information on DTN's Industries:
DTN Oil & Gas | DTN Trading | DTN Agriculture | DTN Weather
Follow DTNMarkets on Twitter
DTN.IQ/IQFeed on Twitter
DTN News and Analysis on Twitter
»Forums Index »IQFeed Developer »IQFeed Developer Support »Regional quotes on no change
Author Topic: Regional quotes on no change (12 messages, Page 1 of 1)

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 17, 2010 01:37 AM          Msg. 1 of 12
I am pretty sure this is a result of 4.7.0.9, as since then the amount of data processed seems to have risen significantly (2-3x). I seem to be getting regional quotes for a market center, and then within a few regional quotes, the exact same regional quote again. As of now almost all of my IQFEED->Application communications are being consumed by regional updates that do not convey any new data, and I know it was not this bad before I upgraded.

Thanks

DTN_Steve_S
-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005


Posted: Feb 17, 2010 08:38 AM          Msg. 2 of 12
I am looking into this. Any information you can provide will be helpful.

Have you downgraded back to the 4.6 and seen the issue go away?
How many regional symbols are you watching (a symbol list would be best)?
Can you quantify the number of updates you receive on 4.6 versus 4.7?
Can you give me an example where you are seeing duplication?

Thanks,

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 17, 2010 09:03 AM          Msg. 3 of 12
I have not yet had a chance to downgrade, but the big giveaway that led me down this path was the amount of data sent from IQFeed to my application. Usually it is around ~10-20gb, with at most ~29gb on a very high volume day. Yesterday ( I upgraded over the weekend) it was ~77gb. The amount of total data sent over the internet connection was the same, however, at around ~3.7gb.

This is a log snippet. The time in front is the time the message was received by my application. It is almost like it is resending the entire regional table every time anything changes.
Quote:
20100217T145523.207862>R,INTC,PSE,20.8000,16200,09:55:10,20.8100,17300,09:55:10,14,4,5,
20100217T145523.207862>R,INTC,PHLX,20.6000,2000,09:55:10,20.9500,1000,09:55:10,14,4,8,
20100217T145523.207862>R,INTC,,20.8000,1700,09:55:10,20.8100,700,09:55:10,14,4,10,
20100217T145523.207862>R,INTC,,20.8000,11800,09:55:10,20.8100,6700,09:55:10,14,4,11,
20100217T145523.207862>R,INTC,,20.8000,2400,09:55:10,20.8100,1400,09:55:10,14,4,12,
20100217T145523.207862>R,INTC,,20.7900,200,09:55:10,20.8200,100,09:55:10,14,4,13,
20100217T145523.207862>R,INTC,,20.8000,100,09:55:10,20.8100,1600,09:55:10,14,4,15,
20100217T145523.207862>R,INTC,,20.8000,3900,09:55:10,20.8100,4300,09:55:10,14,4,16,
20100217T145523.207862>R,INTC,,20.8000,9400,09:55:10,20.8100,6000,09:55:10,14,4,18,
20100217T145523.207862>Q,INTC,20.8000,6413526,100,20.8000,20.8100,17400,17300,09:55:10b,t,
20100217T145523.207862>R,INTC,PSE,20.8000,17400,09:55:10,20.8100,17300,09:55:10,14,4,5,
20100217T145523.207862>R,INTC,PHLX,20.6000,2000,09:55:10,20.9500,1000,09:55:10,14,4,8,
20100217T145523.208350>R,INTC,,20.8000,1700,09:55:10,20.8100,700,09:55:10,14,4,10,
20100217T145523.208350>R,INTC,,20.8000,11800,09:55:10,20.8100,6700,09:55:10,14,4,11,
20100217T145523.208350>R,INTC,,20.8000,2400,09:55:10,20.8100,1400,09:55:10,14,4,12,
20100217T145523.208350>R,INTC,,20.7900,200,09:55:10,20.8200,100,09:55:10,14,4,13,
20100217T145523.208350>R,INTC,,20.8000,100,09:55:10,20.8100,1600,09:55:10,14,4,15,
20100217T145523.208350>R,INTC,,20.8000,3900,09:55:10,20.8100,4300,09:55:10,14,4,16,
20100217T145523.208350>R,INTC,,20.8000,9400,09:55:10,20.8100,6000,09:55:10,14,4,18,
20100217T145523.208350>Q,INTC,20.8000,6413526,100,20.8000,20.8100,17400,17200,09:55:10b,t,
20100217T145523.208350>R,INTC,PSE,20.8000,17400,09:55:10,20.8100,17200,09:55:10,14,4,5,
20100217T145523.208350>R,INTC,PHLX,20.6000,2000,09:55:10,20.9500,1000,09:55:10,14,4,8,
20100217T145523.208350>R,INTC,,20.8000,1700,09:55:10,20.8100,700,09:55:10,14,4,10,
20100217T145523.208350>R,INTC,,20.8000,11800,09:55:10,20.8100,6700,09:55:10,14,4,11,
20100217T145523.208350>R,INTC,,20.8000,2400,09:55:10,20.8100,1400,09:55:10,14,4,12,
20100217T145523.208350>R,INTC,,20.7900,200,09:55:10,20.8200,100,09:55:10,14,4,13,
20100217T145523.208350>R,INTC,,20.8000,100,09:55:10,20.8100,1600,09:55:10,14,4,15,
20100217T145523.208350>R,INTC,,20.8000,3900,09:55:10,20.8100,4300,09:55:10,14,4,16,
20100217T145523.208350>R,INTC,,20.8000,9400,09:55:10,20.8100,6000,09:55:10,14,4,18,
20100217T145523.208350>R,INTC,PSE,20.8000,17400,09:55:10,20.8100,17200,09:55:10,14,4,5,
20100217T145523.208350>R,INTC,PHLX,20.6000,2000,09:55:10,20.9500,1000,09:55:10,14,4,8,
20100217T145523.208350>R,INTC,,20.8000,1700,09:55:10,20.8100,700,09:55:10,14,4,10,
20100217T145523.208350>R,INTC,,20.8000,11800,09:55:10,20.8100,6700,09:55:10,14,4,11,
20100217T145523.208350>R,INTC,,20.8000,2400,09:55:10,20.8100,1400,09:55:10,14,4,12,
20100217T145523.208350>R,INTC,,20.7900,200,09:55:10,20.8200,100,09:55:10,14,4,13,
20100217T145523.208350>R,INTC,,20.8000,100,09:55:10,20.8100,1600,09:55:10,14,4,15,
20100217T145523.208350>R,INTC,,20.8000,3900,09:55:10,20.8100,4300,09:55:10,14,4,16,
20100217T145523.208350>R,INTC,,20.8000,9400,09:55:10,20.8100,5900,09:55:10,14,4,18,



Regional quotes are turned on for:
Quote:
MMM
ADBE
ANF
AES
AET
AMD
A
AFL
ACS
AKS
APD
ABT
ARG
AKAM
AYE
AA
ALL
AGN
ATI
AMZN
MO
AEP
AEE
AXP
AIG
ABC
AMT
AMP
AMGN
APH
APC
ADI
APA
ALTR
AIV
APOL
AAPL
AMAT
ADM
AIZ
T
ADSK
ADP
AN
AZO
AVB
AVY
AVP
BHI
BLL
BAC
BCR
BAX
BBT
BDX
BBBY
BMS
BBY
BIG
BJS
BIIB
BDK
HRB
BMC
BXP
BA
BSX
BNI
BRCM
BMY
CA
COG
CPB
COF
CAH
CFN
CCL
CBG
CELG
CBS
CNP
CEPH
CHRW
CF
CHK
CAM
CVX
CME
CB
CIEN
CI
CAT
CINF
CTAS
CSCO
CTL
CTXS
C
CLX
CMS
COH
KO
CCE
CTSH
CL
CMCSA
CMA
CSC
CPWR
CAG
COP
ED
CNX
STZ
CEG
CVG
GLW
COST
CVH
CSX
CMI
CVS
DHR
DRI
DVA
DF
DE
DELL
DNR
XRAY
DVN
DTV
DO
DFS
D
RRD
DOV
DOW
DV
DPS
DTE
DUK
DNB
DD
DYN
ETFC
EMN
EK
ETN
EBAY
ECL
EIX
EP
ERTS
EMR
EMC
ESV
EOG
EQT
EFX
ETR
EQR
EL
EXPE
EXC
EXPD
ESRX
XOM
FDO
FAST
FII
FDX
FIS
FITB
FHN
FE
FISV
FLIR
FLS
FMC
FTI
FLR
FRX
F
FO
FPL
FCX
FTR
BEN
GME
GCI
GPS
GD
GE
GIS
GNW
GPC
GENZ
GILD
GS
GR
GT
GOOG
GWW
HAL
HOG
HAR
HRS
HIG
HAS
HCP
HCN
HNZ
HSY
HES
HD
HPQ
DHI
HON
HRL
HSP
HST
HCBK
HUM
HBAN
ITW
RX
TEG
ICE
INTC
IPG
IBM
IFF
IGT
IP
INTU
ISRG
IRM
JBL
JEC
ITT
IVZ
JNS
JDSU
JCI
JNJ
JPM
JNPR
KBH
K
KEY
KMB
KIM
KG
KLAC
KSS
KFT
KR
LLL
LH
LM
LEG
LEN
LXK
LUK
LIFE
LLY
LTD
LNC
LLTC
LMT
L
LOW
LO
LSI
M
MTB
MRO
MMC
MI
MAS
MAR
MEE
MA
MAT
MFE
MBI
MKC
MCD
MHP
MCK
MWV
MHS
MDT
WFR
MRK
MDP
MET
PCS
MCHP
MU
MIL
MSFT
MOLX
TAP
MWW
MCO
MON
MOT
MUR
MS
MYL
NBR
NDAQ
NOV
NSM
NTAP
NYT
NWL
NEM
NWSA
NKE
GAS
NI
NBL
JWN
NSC
NU
NTRS
NOVL
NOC
NUE
NVDA
NVLS
NYX
ORLY
OXY
ODP
OMC
ORCL
OI
PCAR
PTV
PLL
PH
PDCO
PAYX
BTU
JCP
PBCT
POM
PBG
PKI
PEP
PFE
PCG
PM
PXD
PNW
PBI
PNC
RL
PPL
PPG
PX
PCP
PCL
PFG
PG
PGN
PGR
PLD
PRU
PEG
PHM
QLGC
PSA
QCOM
DGX
PWR
STR
Q
RSH
RRC
RTN
RHT
RF
RSG
RHI
RAI
ROK
COL
RDC
R
SWY
SNDK
SLE
CRM
SCG
SLB
SCHW
SHLD
SEE
SHW
SIAL
SRE
SNI
SLM
SPG
SJM
SNA
SII
LUV
SO
SWN
SE
S
STJ
SWK
SPLS
SBUX
HOT
STT
SRCL
SYK
SUN
STI
SVU
SYY
SYMC
TROW
TE
TGT
TLAB
THC
TSO
TER
TDC
TXN
TXT
BK
TMO
TIF
TWC
TWX
TIE
TJX
TMK
TRV
TSN
TSS
UNH
UNP
UPS
USB
UNM
VFC
UTX
VAR
VIA
VLO
VMC
VNO
VRSN
VTR
WAG
VZ
WAT
WEC
WDC
WFMI
WFC
WHR
WLP
WIN
WMB
WM
WPI
WPO
WMT
WU
WY
WYN
WYE
WYNN
X
XEL
XLNX
XL
XTO
XRX
YUM
YHOO
ZION
ZMH
ACV
CERN
CHD
DEO
DIS
HBC
HSIC
JOYG
MICC
PBE
PBR
RIG
SY
V
LVS
MGM
MRVL
FSLR
BKS
SPWRA
AONE
DAL
STP
MT
TM
POT
NFLX
BKC
DRYS
XLF
FAZ
FAS
XLE
SDS
SSO
XLV
XLB
XLK
XLI
XLU
QQQQ
SPY
VXX
VXZ
UNG
USO
DBO
UGA
UHN
DDP
SGG
DGL
SRS
SLV
GDX
IAU
GLD

DTN_Steve_S
-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005


Posted: Feb 17, 2010 11:10 AM          Msg. 4 of 12
I have verified this. Unfortunately at this point, I don't think there is a workaround other than simply using version 4.6 for apps that use regional data.

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 17, 2010 02:34 PM          Msg. 5 of 12
If IQFeed gets overloaded does it start dropping data or does it eventually catch up? (While my application can handle the data, do I need to worry about IQFeed getting saturated trying to generate that many regional quotes? Right now it is trying to push 6-7 MB/S peak over one connection).

DTN_Steve_S
-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005


Posted: Feb 17, 2010 02:44 PM          Msg. 6 of 12
If IQConnect is unable to send data to the client app (your app), it will be queued within IQConnect (on your local machine). However, if IQConnect is unable to read data from the server, the server will eventually drop data (it will queue a small amount first but will drop data after a threshold is met) to make sure your connection does not adversly affect other clients connected. IQConnect will prioritize reading data from the server over sending data to a client in order to reduce the amount of data that gets dropped.

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 17, 2010 02:49 PM          Msg. 7 of 12
Is there any way to tell if it is having to drop server data or to up the queue size? It seems to be saturating an entire core which concerns me a bit.

DTN_Steve_S
-DTN Guru-
Posts: 2086
Joined: Nov 21, 2005


Posted: Feb 17, 2010 03:51 PM          Msg. 8 of 12
There is not anyway (yet) in the client to know if data has been dropped. However, if IQConnect is consuming the entire CPU (or 100% of a single core) for an extended period of time, then there is a good chance you are dropping data.

With this amount of data, you will need a fairly high-end machine to process the feed and more importantly you will need extremely efficient data processing within your app (even normal amount of regional messages being sent). Can you give me an idea of the operating environment you are using (CPU/RAM/OS/Internet connection)?

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 17, 2010 04:01 PM          Msg. 9 of 12
Dual Quad Core 2.66ghz, 4gb ram, Windows Server 2008 R2, 6mbit sustained, 12mbit burstable downlink, 1mbit uplink. This usually is not anywhere close to saturated, except for bursts during open/closing. The data processing side of my custom C app has a thread dedicated to decode/dispatch (with the critical sections written mostly using pointer arithmetic) before sending it to other threads for processing.

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Feb 18, 2010 07:34 AM          Msg. 10 of 12
I went ahead and downgraded back down to 4.6.1.0 last night just to be on the safe side, which has nicely taken care of the regional spam and the CPU issue.

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Mar 25, 2010 08:30 PM          Msg. 11 of 12
Any idea when there might be a beta out with a fix for this?

--Thank you

parallelguy
-Interested User-
Posts: 12
Joined: Feb 17, 2010


Posted: Mar 25, 2010 08:58 PM          Msg. 12 of 12
Speaking of which: I found (I believe) another weird oddity, but in 4.6.1.0.

Quote:
20100326T014619.822024>R,IAU,PSE,106.0700,4000000,17:13:01,107.5900,400,17:13:01,14,4,
20100326T014619.822024>R,IAU,NMS,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.822024>R,IAU,BSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.822024>R,IAU,CSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.822024>R,IAU,CHX,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.822024>R,IAU,NYSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.822024>R,IAU,PSE,106.0700,4000000,17:13:01,107.5900,400,17:13:01,14,4,
20100326T014619.823000>R,IAU,NMS,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.823000>R,IAU,BSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.823000>R,IAU,CSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.823000>R,IAU,CHX,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.823000>R,IAU,NYSE,0,0,17:13:01,0,0,17:13:01,14,4,
20100326T014619.823000>R,IAU,PSE,106.0700,4000000,17:13:01,107.5900,400,17:13:01,14,4,




While 4.6.1.0 behaves correctly the vast majority of the time, it appears that when there starts to be zero bidsz/asksz regional entries, it starts sending them even if no updates have occurred. It is unclear to me if this is the same bug reported earlier in the thread. This only seems to be an issue with some stocks too, not all of them.
 

 

Time: Mon August 8, 2022 2:02 PM CFBB v1.2.0 16 ms.
© AderSoftware 2002-2003