Hi,

I'm trying to create 1 minute bar from tick data.
I take tick data with "wAAPL" request.

My logic:
1min bar OPEN - I'll take from field Open, from the first tick at current minute
1min bar HIGH - I'll take from field High , from the tick with Max value at current minute
1min bar LOW - I'll take from field Low, from the tick with Min value at current minute
1min bar CLOSE - I'll take from field Close, from the last tick at current minute
1min bar VOLUME - I'll take from field Last Size, I just sum values from all ticks at current minute

Am I doing write? Maybe you can give me some advices? Or there any thing to optimize in my logic ?

And another question:
Why I can't get tick data on premarket session?
Edited by xvz on Jul 25, 2022 at 03:06 AM

There's already a command that will build live bars for you: Streaming Intervals Bars (BW). Connect to port 9400 and send BW,Symbol,Interval in Seconds,Begin Date/Begin Time. This will give you history up to the current moment, then new incoming ticks as they happen. http://www.iqfeed.net/dev/api/docs//Derivatives_StreamingIntervalBars_TCPIP.cfm goes into more detail.

If you want to build them yourself, there are a few considerations:

1. IQFeed only uses Last-Eligible and Extended ticks to build bars, ignoring Other. The Message Contents field in each ticks will tell you which it is.
2. Volume willl not consistently match the sum of all ticks. This is because Volume can include transactions that aren't ticks, like stock options and implied trades.
3. Make sure you know the difference between "Last" "Most Recent" and "Extended":
- “Last” includes only last -eligible trades.
- “Most Recent” includes all trades.
- “Extended” includes on extended trades (also call Form T or after hours trades).

You would have to consider both "Last" and "Extended" data to re-create bars as IQFeed builds them.

4. You can get tick data on premarket session, but you must be watching the "Extended" fields. "Last" includes only last-eligible trades, which do not occur outside of regular trading hours. These distinctions apply to the fields listed in http://www.iqfeed.net/dev/api/docs/Level1UpdateSummaryMessage.cfm. You can view the one(s) you need.

I hope this is all helpful!

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

Thank a lot!!
This is like bomb for me!
I read all documentation , and from the naming I thought that is only for futures and others.
But now I khow, that I can use it for stocks , and it's really good new!

I am also trying to build interval bars from tick data.
My builds NEARLY match iqfeed interval bars, however there is a mismatch when ticks occur whose time label is on the exact interval boundary end time.

I observe (and expect) that iqfeed intervals are INCLUSIVE of the exact interval start time and I expect NON-inclusive of the interval end time. Both cannot be inclusive because resulting interval bars would overlap.

So an iqfeed 1minute interval bar labeled "09:31:00" should include all ticks from 09:30:00.000000 up through 09:30:59.999999.

But there seems to be a problem that if a tick happens to occur at 09:31:00.000000, it is included also in the iqfeed interval bar labeled "09:31:00". But that tick should instead belong only to the "09:32:00" interval, right?

No, that is correct. Any tick with a timestamp between 9:30:00.000000 and 9:30:00.999999 will be part of the "9:30:00" or "9:30:01" bar.

One other thing I should have mentioned in my prior post: that you can specify whether the label represents the beginning or the end of the time period. It defaults to the end, so "10:40:00" represents 10:39:59 to 10:40:00. If you set LabelAtBeginning to 1 in your HIT/HID/HIX historical request (see http://www.iqfeed.net/dev/api/docs/HistoricalviaTCPIP.cfm) then "10:40:00" will represent 10:40:00 to 10:40:01. This choice does not affect the underlying data in any way, just how it is labeled. A second will consist of ticks with timestamps between .000000 and .999999.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

Thanks Gary.

I'm not completely clear on this comment:
"It defaults to the end, so "10:40:00" represents 10:39:59 to 10:40:00."

The meaning of the text seems to imply that more precisely, it includes only 10:39:59.000000 to 10:39:59.999999. So that at the upper end, "10:40:00" is just a shorthand for saying "10:39:59.999999", regardless of the labelling direction?

And 10:40:00.000000 is in the interval labeled "10:40:01", if the labelling is at the end ?

The point of my question is to make sure that with labelling at the end, I'd have to ask for
"00:00:01" to "24:00:00" to get the whole day. If 24 is an allowed value, otherwise I'd have to take the next days date and "00:00:00".

I'm not quite sure I'm asking this question in the best possible way. I wonder if it's more complicated. Or more simple.
Edited by quickTick on Feb 12, 2023 at 12:37 AM
Edited by quickTick on Feb 12, 2023 at 12:39 AM

I think my previous message needs at least one correction:

I asked: "So that at the upper end, 10:40:00 is just a shorthand for saying 10:39:59.999999, regardless of the labelling direction?"

However I now think that at the upper end, 10:40:00 is a shorthand for 10:39:59.999999 only if the labelling is at the end.

If labelling is at the beginning, it is a shorthand for 10:40:00.999999 .

Strangely that seems to mean that at the lower end, with labelling at the end, 10:40:00 stands for 10:39:59.000000 .

So labelling at the end seems very confusing if one is interested in that detail at all.

The default now seems to be "1" for labelAtBeginning.
For that, the range for the whole day appears to be 00:00:00 to 23:59:59 .
Which will include 23:59:59.999999 .

The specification 00:00:00 to 00:00:00 (next day) would be wrong in either case.

Maybe think of it differently.

For the range [10:40:00 - 10:41:00) - note the difference in brackets. A square bracket means 'includes', a round bracket means 'exclude'.

The best implementation is 'less than'. Anything 'less than' 10:41:00 goes into the 10:40:00 bucket. You then do not need to worry about the irrationality of 10:40:59.999999 as an endpoint.

A 24 hour period would be: [00:00:00 - 00:00:00), that is it starts on midnight, and includes everything up to but not including the next midnight (less than)
Edited by stargrazer on Feb 12, 2023 at 09:02 AM

Quicktick,

You're conflating two things that aren't really related.

"It defaults to the end" has to do with the label a bar has. A bar from 10:40 to 10:45 can be called either 10:40 or 10:45, depending on what LabelAtBeginning is set to. It just affects what time frame is represented.

No matter what label you use, all bars include everything up to .999999 of the specified ending minute and second. A bar from (start time) to (end time) will include all ticks from (start time).000000 to (end time).999999. So 10:40 to 10:45 would include 10:45:00.999999. This is a standard behavior of bars, and how it handles fractions of a second.

I hope this helps.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
Edited by DTN_Gary_Stephen on Feb 13, 2023 at 11:35 AM

I think you meant that 10:40 to 10:45 would include 10:44:59.999999, not 10:45:00.999999, otherwise the following bar will include overlapped data with this bar (trades that occurred in the first second of 10:45:00) and this will cause data inconsistencies.

---

Yair Altman
IQML - IQFeed-MATLAB connector
https://UndocumentedMatlab.com/IQML

I am not a DTN employee; my post reflects my personal opinion
Edited by altmany on Feb 13, 2023 at 11:57 AM
Edited by altmany on Feb 13, 2023 at 12:00 PM

altmany, you're right - I was conflating two things that really weren't related.

I was thinking of historical tick requests, which can include a tick in multiple time ranges. For example:

HTT,EQS,20230210 093035,20230210 093036
LH,2023-02-10 09:30:36.446,1.6800,100,200,1.5900,1.7200,808473395,C,7,3D,0,10,
!ENDMSG!,
HTT,EQS,20230210 093036,20230210 093037
LH,2023-02-10 09:30:36.446,1.6800,100,200,1.5900,1.7200,808473395,C,7,3D,0,10,
!ENDMSG!,

The historical tick request from 9:30:35 to 9:30:36 technically includes everything from 9:30:35 to 9:30:36.999. So A tick at 9:30:36.446 is included in both time ranges.

A HIT request doesn't include the milliseconds after the stop point, because that would put the same data in multiple bars, as you point out. You can see that only one bar contains the 1.68 tick (I picked a low-volume stock for this example):

HIT,EQS,1,20230210 000000,20230210 180000
LH,2023-02-10 09:30:38,1.7200,1.7200,1.7200,1.7200,300,100,0,
LH,2023-02-10 09:30:36,1.6800,1.6800,1.6800,1.6800,200,100,0,
LH,2023-02-10 09:30:00,1.6000,1.6000,1.6000,1.6000,100,100,0,
!ENDMSG!,

I hope I didn't make this even more confusing.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
Edited by DTN_Gary_Stephen on Feb 13, 2023 at 01:01 PM

> A 24 hour period would be: [00:00:00 - 00:00:00), that is it starts on midnight,
> and includes everything up to but not including the next midnight (less than)

That actual implementation appears to be 'less than or equal' for the interval specification.

The following example is from the documentation, asking for the days between the 19th and the 30th. It includes both the 19th and the 30th:

Request: HDT,GOOG,20080919,20080930,,,TESTREQUEST,2500,0<CR><LF>
TESTREQUEST,2008-09-30,425.10,392.32,395.98,400.52,8824587,0,<CR><LF>
TESTREQUEST,2008-09-29,423.51,380.71,419.51,381.00,10764969,0,<CR><LF>
TESTREQUEST,2008-09-26,437.16,421.03,428.00,431.04,5292751,0,<CR><LF>
TESTREQUEST,2008-09-25,450.00,435.98,438.84,439.60,5026917,0,<CR><LF>
TESTREQUEST,2008-09-24,445.00,430.11,430.34,435.11,4242795,0,<CR><LF>
TESTREQUEST,2008-09-23,440.79,425.72,433.25,429.27,5206644,0,<CR><LF>
TESTREQUEST,2008-09-22,454.13,429.00,454.13,430.14,4409119,0,<CR><LF>
TESTREQUEST,2008-09-19,462.07,443.28,461.00,449.15,10008745,0,<CR><LF>
TESTREQUEST,!ENDMSG!,<CR><LF>

So, my current understanding is this:

If you intend to ask for a 24 hour period by using 00:00:00 and 00:00:00 as BeginTime and EndTime, and you do it for two consecutive days, you actually get the 00:00:00 interval twice.

Depending on LabelAtBeginning, you get either the interval before midnight twice, or the intervall after midnight. Although each intervall begins with .000000 and ends with .999999 microsconds. Only the unspecified values are 'less than', whereas the specified interval is 'less than or equal'.

So if you are querying for minute intervals in a 24 hour period, you need to query either
00:00 to 23:59 or
00:01 to 00:00 next day, depending on LabelAtBeginning.

Although in each case, the bar before midnight ends with 23:59:59.999999 , and the bar after midnight ends with 00:00:59.999999.

Please correct if incorrect in any detail.
Edited by quickTick on Feb 14, 2023 at 02:01 PM
Edited by quickTick on Feb 14, 2023 at 02:02 PM
Edited by quickTick on Feb 14, 2023 at 02:04 PM
Edited by quickTick on Feb 14, 2023 at 02:05 PM

If you're requesting tick data, you should request from 000000 to 235959 (or 000001 to 000000) to get an entire 24-hour day today with no overlap. This can be demonstrated with an example:

HTT,QCL#,20230214 000000,20230215 000000
LH,2023-02-15 00:00:00.788827,78.37,1,11813,78.37,78.38,11887050,C,112,01,2,15,
LH,2023-02-15 00:00:00.788827,78.37,1,11812,78.37,78.38,11887050,C,112,01,2,15,p
LH,2023-02-14 23:59:53.336951,78.38,1,11809,78.38,78.39,11887046,C,112,01,2,15,
LH,2023-02-14 23:59:40.433622,78.38,1,11808,78.38,78.39,11887042,C,112,01,2,15,
(lots of ticks)
LH,2023-02-14 00:00:25.045924,79.26,1,11300,79.26,79.27,11415859,C,112,01,2,14,
LH,2023-02-14 00:00:02.727061,79.26,2,11299,79.25,79.26,11415857,C,112,01,1,14,
LH,2023-02-14 00:00:00.784311,79.25,1,11297,79.25,79.26,11415852,C,112,01,2,14,
LH,2023-02-14 00:00:00.784311,79.25,1,11296,79.25,79.26,11415852,C,112,01,2,14,
LH,2023-02-14 00:00:00.755201,79.26,1,11294,79.26,79.27,11415849,C,112,01,2,14,
LH,2023-02-14 00:00:00.755201,79.26,1,11293,79.26,79.27,11415849,C,112,01,2,14,
!ENDMSG!,

As you can see, a request from 000000 to 000000 includes the first second after midnight for both days. A request from 000000 to 235959 includes up to 235959.999999, but nothing after 000000.0000000.

HTT,QCL#,20230214 000000,20230215 235959
LH,2023-02-14 23:59:53.336951,78.38,1,11809,78.38,78.39,11887046,C,112,01,2,15,
LH,2023-02-14 23:59:40.433622,78.38,1,11808,78.38,78.39,11887042,C,112,01,2,15,
(etc)
!ENDMSG!,

Again, this only applies to tick-level requests. Bars will not overlap in this way:

HIT,QCL#,60,20230214 235700,20230215 000300
LH,2023-02-15 00:03:00,78.37,78.35,78.36,78.36,11878,30,0,
LH,2023-02-15 00:02:00,78.37,78.35,78.36,78.36,11842,7,0,
LH,2023-02-15 00:01:00,78.37,78.37,78.37,78.37,11833,6,0,
LH,2023-02-15 00:00:00,78.37,78.35,78.37,78.36,11822,9,0,
LH,2023-02-14 23:59:00,78.38,78.36,78.36,78.38,11809,19,0,
LH,2023-02-14 23:58:00,78.36,78.34,78.35,78.34,11785,19,0,
LH,2023-02-14 23:57:00,78.35,78.35,78.35,78.35,11761,6,0,
!ENDMSG!,

The 00:00:00 bar includes only the ticks from 00:00 to 00:00.999999.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
Edited by DTN_Gary_Stephen on Feb 15, 2023 at 08:43 AM

Within a single request, there is no overlap.

I was going by your statement "If you intend to ask for a 24 hour period by using 00:00:00 and 00:00:00 as BeginTime and EndTime, and you do it for two consecutive days." In this scenario, if it is an HTT request, you would get the first second after midnight as part of both requests. HIT requests will not have this problem.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

Your example doesn't show an HIT request using 00:00:00 and 00:00:00 as BeginTime and EndTime. Only the example HTT request does so.

So far, I would expect two HIT requests for two consecutive days, using 00:00:00 and 00:00:00 as BeginTime and EndTime, to both include the 00:00:00 bar that is between these two days. I believe this according to the example in the documentation.

However within a single request that crosses midnight, I would not expect any problem within the middle of the request, as your example shows. I would not expect such a problem for HTT either. The bars within a single request never overlap. Or if they did, that would a bug and completely different. I don't think we have a bug here, just that the documented behavior is easy to misunderstand when deciding which labels to use for BeginTime and EndTime, if you actually want the first bar to be *after* the BeginTime label, and the last bar to be *before* the EndTime label, which is spresumably the case if you use 00:00:00 to 00:00:00 intending to get 24 hour period. Since then, depending on the value of LabelAtBeginning, both the first and the last bar will be either before, or after each midnight.

If so, there might be a value in having a third option for LabelAtBeginning, where the label for BeginTime would be at beginning, and the label for EndTime would be at end.

I'm having some trouble which I think is related to this topic. I'm making a HIT request for @ES#, 60 second interval.

I notice that 0:00:00 is missing for the day and after-hours trading starts at 18:01:00. This would be consistent with LabelAtBeginning being set to 0, but I tried using the default (which should be 1) and explicitly setting this argument to 1, and neither changed the result.

Am I doing something wrong? What am I missing?

The only possibility is normal for intervals to be missing if no trading occurred during that time period. You don't mention what day you inquired about, but it's possible there were no trades between 00:00-00:01 and 18:00-18:01 on that day. If you post the request here, I can dig deeper.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist

I basically see it every day:

HIT,@ES#,60,20221101,20221101,,,,,,,0\r\n

Here's the beginning of the reply, 0:00 is missing...

datetime high low open last volume
11/1/2022 0:01 3898 3897.5 3897.5 3897.75 63
11/1/2022 0:02 3897.75 3896.75 3897.75 3897 66
11/1/2022 0:03 3897.25 3896.5 3897 3896.75 60
11/1/2022 0:04 3896.75 3896.25 3896.75 3896.5 72


And here's the response around the transition to overnight trading. 17:00 has a trade, which it shouldn't if we're labeling beginning of period, and 18:00 is missing.

datetime high low open last volume
11/1/2022 16:59 3862.25 3861.75 3862 3862.25 255
11/1/2022 17:00 3862.25 3861.75 3862.25 3861.75 261
11/1/2022 18:01 3865.75 3862.75 3863 3864.25 539
11/1/2022 18:02 3864.5 3863.25 3864.5 3863.25 116

Did you notice your command returns this:

HIT,@ES#,60,20221101,20221101,,,,,,,0
2022-11-02 00:00:00,3875.50,3875.25,3875.50,3875.25,45513,47,
2022-11-01 23:59:00,3875.75,3875.25,3875.50,3875.50,45466,32,
(etc)
2022-11-01 00:03:00,3897.25,3896.50,3897.00,3896.75,51631,60,
2022-11-01 00:02:00,3897.75,3896.75,3897.75,3897.00,51571,66,
2022-11-01 00:01:00,3898.00,3897.50,3897.50,3897.75,51505,63,
!ENDMSG!,

You're using LabelAtBeginning=0. which means the 0:01 bar represents 0:00 to 0:01. When you request all 1-minute bars for a day und use LabelAtBeginning=0, the last minute of November 1 is called "11-02 0:00." You're not getting the 0:00 bar you're looking for, because it's technically part of the previous day.

Also, you must be using a very old protocol, because modern IQFeed syntax will not allow you to specify a date in the HIT command without also specifying a time.

HIT,@ES#,60,20221101,20221101,,,,,,,0
2022-11-02 00:00:00,3875.50,3875.25,3875.50,3875.25,45513,47,
2022-11-01 23:59:00,3875.75,3875.25,3875.50,3875.50,45466,32,
2022-11-01 23:58:00,3876.00,3875.50,3875.50,3875.50,45434,37,
(etc)
2022-11-01 00:03:00,3897.25,3896.50,3897.00,3896.75,51631,60,
2022-11-01 00:02:00,3897.75,3896.75,3897.75,3897.00,51571,66,
2022-11-01 00:01:00,3898.00,3897.50,3897.50,3897.75,51505,63,
!ENDMSG!,

S,SET PROTOCOL 6.2
S,CURRENT PROTOCOL,6.2

HIT,@ES#,60,20221101,20221101,,,,,,,0
E,Invalid Parameter - IntervalType must be 't' or 's' or 'v'.,
!ENDMSG!,

"IntervalType must be" is one of the many differences between the default protocol 4.9 and the current 6.2. If I do this correctly using the modern protocol:

HIT,@ES#,60,20221101 000000,20221101 235900,,,,,,,s,0

I get the same result.

LH,2022-11-02 00:00:00,3875.50,3875.25,3875.50,3875.25,45513,47,0,
LH,2022-11-01 23:59:00,3875.75,3875.25,3875.50,3875.50,45466,32,0,
(etc)
LH,2022-11-01 00:02:00,3897.75,3896.75,3897.75,3897.00,51571,66,0,
LH,2022-11-01 00:01:00,3898.00,3897.50,3897.50,3897.75,51505,63,0,
!ENDMSG!,

As for the second example, it looks like there *is* a 17:00 bar most days for @ES#. I can't view the underlying ticks for your example, because 11/1/2022 is more than six months ago. If you could post the entire command, that would help.

---

Sincerely,
Gary Stephen
DTN IQFeed Implementation Support Specialist
Edited by DTN_Gary_Stephen on Apr 6, 2023 at 02:21 PM

Thanks! It was the older protocol tripping me up, and a stupid mistake. I was setting the protocol on the admin port but not the history port.

I actually want LabelAtBeginning=1, the 0 in my command was an error. I initially left this parameter off, because I read the default was LabelAtBeginning=1. But I guess the default changed between protocol versions, which is why I later manually specified it (but then did so incorrectly).

Now to fetch a day's data with 0:00 being the first bar, I'm doing:

HIT,@ES#,60,20221101 000000,20221201 000000,,,,,,,s,1\r\n

Which is giving me what I want.