I have a question regarding how IQFeed handles concurrent requests for data. Basically, is it possible to make several requests to the History interface and will IQFeed, given multiple requests, attempt to download all of them concurrently?


There are 2 options I am thinking of:


Option 1:

1. Start IQFeed
2. Start up x threads.
3. For each thread,:
a. connect to IQFeed's TCP/IP history port
b. request data




Option 2:

1. Start IQFeed
2. Start up x threads
3. For each thread:
a. Use IHistoryLookup2 (or IHistoryLookup1) to request data



Will either of these work? I mean, I know I can do #1 but I haven't been able to tell if it's concurrent yet as I haven't stressed it enough yet (and last I worked with IQFeed was months ago).

Any advice?

Here is what I do:

1) Start IQFeed
2) Start 1 thread
3) Use Overlapped IO to open a fixed number of ports to TCP/IP history port
4) accept input from multiple ports into multiple buffers
5) as you process the end of input on a port, reuse the port and send another history command
6) as request list ends, close out ports and buffers.

This shouldn't be done during trading hours.

There is a caveat. I've written this code in both C# and C++. I've found that after downloading a bunch of symbols, I'll see that one or two requests don't finish. With my recent C++ code, I'm hoping to put together a small demo and upload to the developers so they can test for the dropped connection. I saw this happen last night even after installing the new 4.4 beta.

Just to confirm what stargrazer said, yes, IQFeed will make concurrent requests to the servers in order to fill multiple requests at the same time. Both option 1 and option 2 should be possibilities.

stargrazer, I haven't run into the same issues internally. Let me know when you have a test app ready and I'll see if I can figure out what might be causing your issues.

Excellent. Thanks!

I have another similar question regarding History data over TCP/IP.


If I request History data over TCP/IP, then close the port to IQFeed prematurely, is any bandwidth saved or has IQFeed already downloaded all the data before it begins to stream it over the local port?

IQFeed starts streaming the data to your application as soon as it is received from the server. However, I do not believe any bandwidth will be saved. The request from the server will still complete, and IQFeed will simply thow away any data that is received and undeliverable.

Ok. Thanks.

Steve,

Have you tried running simultaneous downloads in a bandwidth constricted scenario? For example, on a dsl line, I try to keep the line clear during data transfer, but sometimes end up downloading some file or other. It is at that time that some symbols don't complete: when bandwidth of download requests lead to dropouts (or at least it appears that way).

I'm getting closer to getting my project split into libraries so it will be easier to get a test scenario sent to you.

I have tested this scenario briefly using software based badwidth throttling to limit the machines bandwidth but probably not any testing that would directly duplicate your scenario.

Can you give me a better idea of what circumstances you are operating under?

What is the speed of your DSL (up/down)?
How many concurrent requests?
How much data is being requested in each one?

I have 640K down, 256K up. I have 40 concurrent HD requests for the latest 20 bars. I cruised through 8700 equity symbols on Amex, NMS, NYSE. Most runs are ok, but last night with another download occurring at the same time, three symbols out of 8700 did not complete. They were three random symbols starting with P. So it wasn't a beginning of set problem. I'll see if I can recreate it again tonight and see if my data structures actually received anything.

Tonight: 8718 symbols requested. HD,XXX,1; 40 overlapping requests. The port is reused for another request after each request is complete. 1 bar for symbol TPX requested. No bars recieved. My response handling state machine indicates that after the request, no responses of any sort were received.

I did a second run with the only change being obtain the latest 20 bars. It ran to completion. But I did notice on further look of messages, I get error messages like '!ERROR! Invalid symbol.' on symbols that do have content: OLN, OKE, ... There are many symbols like this. Which leads me to running the download twice to get all the symbols, as it doesn't come up the same each time.

I do notice that when doing the same symbols in a telnet session, there may be a delay before data comes back. Which is fine. But if there is a long delay, is that when the 'Invalid Symbol' message gets generated somehow? And if there is a really long delay, perhaps there is no message, and that is why I get one or two symbol requests that never get a response.