Hello forum,

I am new to the IQFeed api.
I have been trying to find the proper documentation on how to usd the IQFeed API, but wasn't able to find one yet.
(The only source I got is the YouTube video from IQFeed, which doesn't show more in-depth information, and the pages shown in the video are not accessible now.)

1. Is there a relatively new and most up to date documentation to refer for developing with IQFeed API?

2. The main goal for me at the moment is to access the realtime Level 2 market depth data for Futures markets (i.e. CME Group) with Python programming language.

I have seen the IQFeed.CSharpApiClient for Python by mathpaquette on Github,
and was able to execute the sample python code which was uploaded in the repo: https://github.com/mathpaquette/IQFeed.CSharpApiClient/blob/master/docs/USING-WITH-PYTHON.md

However, I have no idea on where to start looking at to implement the Level 2 data streaming with this repo.

- Does anyone have a basic sample code for using the Streaming.Level2 api with IQFeed.CSharpApiClient?

- If not, can anyone guide me to some starting points on how to use the IQFeed.CSharpApiClient in python?

Currently, I am stuck at this level:

# Python v3.7.7
# Windows 10 x64
import sys
import clr

assembly_path = r"~my_path_to\IQFeed_CS"
sys.path.append(assembly_path)
clr.AddReference("IQFeed.CSharpApiClient")

from IQFeed.CSharpApiClient import IQFeedLauncher
from IQFeed.CSharpApiClient.Streaming.Level2 import Level2ClientFactory
from IQFeed.CSharpApiClient.Streaming import Level2
from IQFeed.CSharpApiClient.Streaming.Level2 import Messages
from IQFeed.CSharpApiClient.Streaming.Common import Messages


IQFeedLauncher.Start("my_id", "my_pw", "dev_id")

level2Client = Level2ClientFactory.CreateNew()

level2Client.Connect()

level2Client.ReqWatch("@BTC#")

...

Thank you in advance.

Sincerely,

Andrew.

Andrew,

Hello, this is Gary Stephen from DTN IQFeed API Support.

The online documentation about the API is available at http://www.iqfeed.net/dev/index.cfm?login=login. You will need a username and password to access it, which you would have received when you purchased the IQFeed API product.

The code you give appears to import some necessary packages, connect to the API, and request a watch for a certain symbol.

IQFeedLauncher.Start("my_id", "my_pw", "dev_id") performs the act of connecting to the server, as per http://www.iqfeed.net/dev/api/docs//InitializingTheFeed.cfm. You would have to set these variables to your own login, password, and productid.

level2Client = Level2ClientFactory.CreateNew() creates a new instance.

level2Client.Connect() connects to the socket, as described in http://www.iqfeed.net/dev/api/docs/Introduction.cfm.

level2.Client.ReqWatch sends the command "w@BTC#" which is one of the commands outlined in http://www.iqfeed.net/dev/api/docs//Level2viaTCPIP.cfm. Each socket has its own set of commands.

In general, the best way to understand an existing IQFeed library is to realize which part does what. Hopefully this will help. I can answer any further questions you may have about it.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

Hello Andrew,

Feel free to create an issue https://github.com/mathpaquette/IQFeed.CSharpApiClient/issues if you need more support.

Thanks,
Math

Hello Andrew,

Added 2 additional examples for Python and one is related to what you were asking for.
Please have a look to Streaming level 1 example.

https://github.com/mathpaquette/IQFeed.CSharpApiClient/blob/master/docs/USING-WITH-PYTHON.md#additional-examples

Regards,
Mathieu Paquette
Software consultant

hi,

I got the streaming level 1 working and it is pumping out messages like (see below)

I am using the code posted in the example Docs (Streaming Level 1). So I will be able to handle these data but I also need the tickID. This is not included in this particular message rule. Would you be so kind to show how I add the tickID to a message rule and maybe also show how we could assemble a message rule. Since I do not need the text. Just the comma separated data would do.

Is this possible or would I have to change CSHARP code itself and compile etc.

thank you

Ed


Symbol: QGC#, MostRecentTrade: 1652.2, MostRecentTradeSize: 1, MostRecentTradeTime: 04:58:50.0467290, MostRecentTradeMarketCenter: 113, TotalVolume: 53961, Bid: 1652.2, BidSize: 4, Ask: 1652.3, AskSize: 4, Open: 1669, High: 1669.1, Low: 1649.3, Close: 1670, MessageContents: a, MostRecentTradeConditions: 01
Symbol: QGC#, MostRecentTrade: 1652.2, MostRecentTradeSize: 1, MostRecentTradeTime: 04:58:50.0467290, MostRecentTradeMarketCenter: 113, TotalVolume: 53961, Bid: 1652.2, BidSize: 4, Ask: 1652.3, AskSize: 5, Open: 1669, High: 1669.1, Low: 1649.3, Close: 1670, MessageContents: a, MostRecentTradeConditions: 01
Symbol: QGC#, MostRecentTrade: 1652.2, MostRecentTradeSize: 1, MostRecentTradeTime: 04:58:50.0467290, MostRecentTradeMarketCenter: 113, TotalVolume: 53961, Bid: 1652.2, BidSize: 4, Ask: 1652.3, AskSize: 6, Open: 1669, High: 1669.1, Low: 1649.3, Close: 1670, MessageContents: a, MostRecentTradeConditions: 01

nevermind, I succeeded I think

i used

from IQFeed.CSharpApiClient.Streaming.Level1 import Level1ClientFactory
from IQFeed.CSharpApiClient.Streaming.Level1 import DynamicFieldset
from IQFeed.CSharpApiClient.Streaming.Level1.Handlers import Level1MessageDynamicHandler

and then

level1Client = Level1ClientFactory.CreateNew(Level1MessageDynamicHandler())

and then I could add the fields I want via

fields = [DynamicFieldset.Symbol,
DynamicFieldset.LastTime,
DynamicFieldset.Last,
DynamicFieldset.LastSize,
DynamicFieldset.TotalVolume,
DynamicFieldset.Bid,
DynamicFieldset.Ask,
DynamicFieldset.TickID]
level1Client.SelectUpdateFieldName(fields)

seems to work ok. Will test when markets are open

hi, i was wondering if someone is using this csharp plugin.

I have been playing around with it but I have no developer account at IQFeed so a lot of trial and error. I have been successful in using Python with this plugin. Both the historical data part and the streaming data part.

At the moment I am optimizing the historical part since it is really slow. Slow in comparison to the speed in which Amibroker downloads the same number of data. I guess 1 variable could influence this: "datapointsPerSend"

what is considered the best size for this. In the source I could not find the default size other than "null". What is the optimal size with respect to speed? 2048? I have no clue what numbers I need to use in here.

Thank you

i figured out that not the reading of the data takes a long time but putting them in a dataframe is the problem.

so below some simple testcode. So the problem is that reading all the tick message lines and put them in a dataframe is the problem. So up until this line all goes superfast:

 
# Read ticks from disk
ticksFromFile = TickMessage.ParseFromFile(dstTicksFilename)

After that, the loop readings the tick lines is very slow. How would I program this part more efficiently? I used the example from the csharp github example.

thanks

import sys
import clr
assembly_path = r'C:/Program Files/AmiBroker/IQFeedCSharpApiClient'
sys.path.append(assembly_path)
clr.AddReference("IQFeed.CSharpApiClient")
import time
import numpy as np
import pandas as pd
from IQFeed.CSharpApiClient.Lookup import LookupClientFactory
from IQFeed.CSharpApiClient.Lookup.Historical.Messages import TickMessage
import os

# Create Lookup client
lookupClient = LookupClientFactory.CreateNew()

# Connect
lookupClient.Connect()

# Save ticks to disk
ticksFilename = lookupClient.Historical.File.GetHistoryTickDatapoints("AAPL", 10000, 1)

# Move tmp filename
dstTicksFilename = "ticks.csv"
os.replace(ticksFilename, dstTicksFilename)

# Read ticks from disk
ticksFromFile = TickMessage.ParseFromFile(dstTicksFilename)

headers = ['timestamp','month','day','year','hour','minute','second','last','volume','totalvolume','bid','ask','id']
df = pd.DataFrame(columns=headers)
for tick in ticksFromFile:
    trade={ 'timestamp':tick.Timestamp.ToString("yyyy-MM-dd HH:mm:ss.ffffff"),
            'month':int(tick.Timestamp.Month),
            'day':int(tick.Timestamp.Day),
            'year':int(tick.Timestamp.Year),
            'hour':int(tick.Timestamp.Hour),
            'minute':int(tick.Timestamp.Minute),
            'second':int(tick.Timestamp.Second),
            'last':float(tick.Last),
            'volume':int(tick.LastSize),
            'totalvolume':int(tick.TotalVolume),
            'bid':float(tick.Bid),
            'ask':float(tick.Ask),
            'id':int(tick.TickId)}
    df = df.append(trade, ignore_index=True)

#df = df[::-1] # reverse order
df.index = pd.to_datetime(df['timestamp'])
df.drop(df.loc[df['volume']==0].index, inplace=True)

i couldn't do any further testing since IQFeed historical data seems to be down all afternoon. I can make contact to IQFeed but it refuses to send historical data.

I was thinking about my previous post. It gets the historical data and puts it in a file. It should be able to put a CSV file into a Pandas Dataframe in 1 go. Not line by line. That is the reason it is so slow. I have done that for @ES# tick files of like 800000 lines long ( 1 day of data) and reads it into a pandas dataframe just like that, very fast. So I need to figure out how to get access to the "ticks.csv" file.

Or in the first example shown here:

https://github.com/mathpaquette/IQFeed.CSharpApiClient/blob/master/docs/USING-WITH-PYTHON.md


I need to be able to read "ticks" directly into a pandas dataframe, not line by line.

ticks = lookupClient.Historical.GetHistoryTickDatapoints("AAPL", 100)

once in the Dataframe I can change the columns if necessary.

already found this "ticks.csv" file. So this can be read at once into pandas and is very fast. Pretty excited about this.

Unfortunately the server is still down. At least I can't get historical data. Looks like everyone at IQFeed is on holiday. Is there no STATUS page or something like that? Since it is down now for more than 24 hours. I know it is weekend but some people need to write code in the weekends. Strange, seems like everybody left the shop. Guess they will show up around 9AM ET tomorrow (if we are lucky).

hi,

i wonder if someone else is playing around with this. So I am already making good progress. From the examples (see csharp manual by mathpagquette) I can get for instance 1000 lines of data using:

 
ticksFilename = lookupClient.Historical.File.GetHistoryTickDatapoints("@ES#", 1000, 1)

also I succeeded getting a certain number of days using, e.g.

ticksFilename = lookupClient.Historical.File.GetHistoryTickDays("@ES#",3,None,None,None,1)

I am however having difficulties getting tick data from a begindate to an enddate.

In the source the format is given as:

 
        public string GetHistoryTickTimeframe(string symbol, DateTime? beginDate, DateTime? endDate, int? maxDatapoints = null,
            TimeSpan? beginFilterTime = null, TimeSpan? endFilterTime = null, DataDirection? dataDirection = null,
            string requestId = null, int? datapointsPerSend = null)
        {
            return GetHistoryTickTimeframeAsync(symbol, beginDate, endDate, maxDatapoints, beginFilterTime,
                endFilterTime, dataDirection, requestId, datapointsPerSend).SynchronouslyAwaitTaskResult();
        }

so I tried:

 
ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",20221013,20221014,None,None,None,1)

but it rejects this. I tried various alternatives but it doesn't accept it. Could someone show the proper format? I use the same dateformat as IQFeed uses. I tried passing it as a string but no success

Thanks

hi, in response to an email from Gary:

I'd need to know what command mathpaquette is trying to send, and what failure message you're getting. An iqconnect log file would be helpful. To generate one, open the Diagnostics app and go to the Logging tab. Make sure "lookup request" "lookup data" and "lookup error" are all checked. Try making the request again in mathpaquette. then go back to the diagnostics tab and collect the log file. You can email it to support or post it here, and I can review it.

I add the diagnostics file but it seems no info there on the error. So I explain what I do. The IQFeed connection manager is launched via Amibroker. So I test the code in the Python 3.8 console.

First I load into the console:

import sys
import clr
assembly_path = r'C:/Program Files/AmiBroker/IQFeedCSharpApiClient'
sys.path.append(assembly_path)
clr.AddReference("IQFeed.CSharpApiClient")
import time
import numpy as np
import pandas as pd
from IQFeed.CSharpApiClient.Lookup import LookupClientFactory
from IQFeed.CSharpApiClient.Lookup.Historical.Messages import TickMessage
from datetime import datetime

# Create Lookup client
lookupClient = LookupClientFactory.CreateNew()
# Connect
lookupClient.Connect()
 

so in the console what I did looks like:

 
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import clr
>>> assembly_path = r'C:/Program Files/AmiBroker/IQFeedCSharpApiClient'
>>> sys.path.append(assembly_path)
>>> clr.AddReference("IQFeed.CSharpApiClient")
<System.Reflection.RuntimeAssembly object at 0x000002560C99BDC0>
>>> import time
>>> import numpy as np
>>> import pandas as pd
>>> from IQFeed.CSharpApiClient.Lookup import LookupClientFactory
>>> from IQFeed.CSharpApiClient.Lookup.Historical.Messages import TickMessage
>>> from datetime import datetime
>>>
>>> # Create Lookup client
>>> lookupClient = LookupClientFactory.CreateNew()
>>> # Connect
>>> lookupClient.Connect()
>>>
>>>
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickDatapoints("@ES#",100,1)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickDays("@ES#",1,None,None,None,1)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#","20221013 093001","20221014 093020")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No method matches given arguments for GetHistoryTickTimeframe: (<class 'str'>, <class 'str'>, <class 'str'>)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#","20221013","20221014",None,None,None,1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No method matches given arguments for GetHistoryTickTimeframe: (<class 'str'>, <class 'str'>, <class 'str'>, <class 'NoneType'>, <class 'NoneType'>, <class 'NoneType'>, <class 'int'>)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No method matches given arguments for GetHistoryTickTimeframe: (<class 'str'>)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",None,None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
System.ArgumentException: Begin date or End date must have value.
   at IQFeed.CSharpApiClient.Lookup.Historical.Facades.HistoricalFileFacade.GetHistoryTickTimeframe(String symbol, Nullable`1 beginDate, Nullable`1 endDate, Nullable`1 maxDatapoints, Nullable`1 beginFilterTime, Nullable`1 endFilterTime, Nullable`1 dataDirection, String requestId, Nullable`1 datapointsPerSend)
>>>

as you can see I use 2 of the 3 tickMessage functions succesfully, these 2:

 
ticksFilename = lookupClient.Historical.File.GetHistoryTickDatapoints("@ES#",100,1)
ticksFilename = lookupClient.Historical.File.GetHistoryTickDays("@ES#",1,None,None,None,1)

there is a third function in which you can add the beinDate/Time endDate/Time. I tried all kinds of formats. For this error log I tried these 4:

 
ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#","20221013 093001","20221014 093020")
ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#","20221013","20221014",None,None,None,1)
ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#")
ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",None,None)

as you can see from the 4-th error message in the console it knows the function since the error it gives is:

 
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",None,None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
System.ArgumentException: Begin date or End date must have value.
   at IQFeed.CSharpApiClient.Lookup.Historical.Facades.HistoricalFileFacade.GetHistoryTickTimeframe(String symbol, Nullable`1 beginDate, Nullable`1 endDate, Nullable`1 maxDatapoints, Nullable`1 beginFilterTime, Nullable`1 endFilterTime, Nullable`1 dataDirection, String requestId, Nullable`1 datapointsPerSend)
>>>

I can see the 2 successful requests in the log, but not the last 4 unsuccessful. Going through the csharp source code I am pretty sure the format for the beginDate and endDate needs to be like:

"yyyyMMdd HHmmss"

And I also have other pure Python code that just passes the beginDate and endDate as strings in which I use (and this actually WORKS):

fdate = next_day.strftime("%Y") + next_day.strftime("%m") + next_day.strftime("%d") # format YYYYMMDD
message = "HTT,%s,%s,%s,,,,1\n" % ( sym, fdate, fdate )

I tried about everything so I hope someone can figure it out.

thanks

and I know the endDate and beginDate are supposed to be of the DateTime type. I also tried this, see console printout below. Again I get a type error. So it does not accept "datetime" types and also no "str" types. Also no "int" types ....

Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import clr
>>> assembly_path = r'C:/Program Files/AmiBroker/IQFeedCSharpApiClient'
>>> sys.path.append(assembly_path)
>>> clr.AddReference("IQFeed.CSharpApiClient")
<System.Reflection.RuntimeAssembly object at 0x000002E7DA40BDC0>
>>> import time
>>> import numpy as np
>>> import pandas as pd
>>> from IQFeed.CSharpApiClient.Lookup import LookupClientFactory
>>> from IQFeed.CSharpApiClient.Lookup.Historical.Messages import TickMessage
>>> import datetime
>>> from System import DateTime
>>>
>>> # Create Lookup client
>>> lookupClient = LookupClientFactory.CreateNew()
>>> # Connect
>>> lookupClient.Connect()
>>> d1 = datetime.date(2022, 10, 13)
>>> d2 = datetime.date(2022, 10, 14)
>>> d1
datetime.date(2022, 10, 13)
>>> d2
datetime.date(2022, 10, 14)
>>> ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",d1,d2,None,None,None,1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: No method matches given arguments for GetHistoryTickTimeframe: (<class 'str'>, <class 'datetime.date'>, <class 'datetime.date'>, <class 'NoneType'>, <class 'NoneType'>, <class 'NoneType'>, <class 'int'>)
>>>

well I made some progress. At least it seems to accept the type: <class 'System.DateTime'>

so this code at least makes the function work. So making some progress. I know the type it needs now. Still getting some weird results but I have something to work with now

import clr
from pandas import to_datetime, Series
clr.AddReference('System.Collections')
from System.Collections.Generic import List
from System import DateTime
Contracts = to_datetime(Series(['13/10/2022','14/10/2022']))

DateList = List[DateTime](range(len(Contracts)))
for i in range(len(Contracts)):
    DateList.Add(DateTime(Contracts.year,Contracts.month,Contracts.day))

d1 = DateList.get_Item(0)
d2 = DateList.get_Item(1)

ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe("@ES#",d1,d2,None,None,None,1)

Edited by emp on Oct 18, 2022 at 03:52 AM

to finish off my message marathon here is the solution and I have to so say it works great

So when loading Python you have to additionally load:

 
    clr.AddReference('System.Collections')
    from System import DateTime

then to get the data between 2 dates is done by using for instance

 
    d1 = DateTime(2022,10,17,9,10,10)
    d2 = DateTime(2022,10,18,8,0,0)
    ticksFilename = lookupClient.Historical.File.GetHistoryTickTimeframe(sym,d1,d2,None,None,None,1)

so the parameters used in DatetTime are Year, Month, Day, Hour, Minute, Second
Edited by emp on Oct 18, 2022 at 08:00 AM

That makes sense. If the date wasn't being built correctly (in the format YYYYMMDD) then an HTT tick history request would fail.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist