jay
i am getting big spikes in the iq feed that goes into the new qt historical charts.rendering the charts unuseable.
symbols are
bgbpusd
busdchf
busdcad
bgbpjpy
teurjpy
bnzdusd
beurgbp
baudusd
dxy$y
thanks

also-posted-on-quotetracker-website.who-is-able-to-correct-this?qt-or-iq.thanks

We will try to get these cleaned up so they look better on charts. Unfortunately, the forex data providers don't do much to ensure quality data. We are looking at putting our Radical Price checking into place on this data, but because it is an OTC market, the forex brokers tell us the data is all valid (with the exception of zero values). Of course, we realize this wreaks havok on the charts, so we will do our best to scrub it and clean things up.

---

Jay Froscheiser
DTN - Trading Markets

thanks jay.
an easy alternative is to insert a new price near the high/low of that day so it does not screw up the historical charts.all we are looking for is the general trend on the historical charts.as far as i can see,on most charts it is only ONE bad bit of data.ie on bgbpusd it is only the high on 29/07/05,so it will not interfere too much with the general trend if the high is only a few points out
thanks

jay.hopefully you had a great thanksgiving weekend.we do not celebrate this in england,the pilgrim fathers were persecuted here.
will the data clean up happen soon?
thanks

jay.thanks.looks like you have cleaned up the data.i think you might have missed bgbpjpy?
also what are the symbols for 5 year bill and 2 year bill.are ther futures for these? if so what are the symbols?
thanks again

a lot of gaps appearing on the historical spot fx charts on quotetracker.according to jerry medved,the problem is wth iq and not quotetracker.will it be fixed soon.thanks.also any update on adding the ftse 250 future from liffe

are the gaps you are seeing on daily data? If you can narrow down where you are seeing the issues, we can get our market data group to work on cleaning it up. We are hoping to get the radical price checking running on our forex feeds in the next couple weeks. In addition we have a new server release in QA for the IQ system that will do a better job at correcting bad data on the fly. That is 1-2 months out.

---

Jay Froscheiser
DTN - Trading Markets

jay..gaps are on the daily charts for barclays and tullet fx symbols.dating from 15/01/06 to today.symbols are bgbpusd,beurusd etc etc.thanks

I looked at both charts on QuoteTracker with IQ Feed and didn't see any gaps in data or range issues. Can you give me a specific example of a high/low issue that causing bad ticks on the chart.

i will email a chart in a couple of hours.just about to go and have supper.it is 6 pm in london.i will email it to support@iqfeed.net

i have sent the chart.however, you can check out the data with another charting program at www.netdania.com.you will see there are big differences.thanks

I don't see gaps but I do see some range issues....Let me look into this further and I will get back to you.

gaps are my expression for range issues.you can double check the data with netdania for all the major crosses.thanks

The Forex spots you sent us do have some range issues. The IQFeed Development team is working on some updates that would address these problems.

Could someone go into a little more detail about how the data clean up will affect realtime vs archival data? I'd like archival data to reflect what I'll be getting realtime. The archival needs to be this way so that I know what I've simulated on archival will happen on real time. If I tune on cleaned archival data, and run on uncleaned real time, then my trading rules won't be consistent.

I have various things in place to ignore the outliers and is consistent during simulation as well as real time.

Thanx.

The historical data clean up won't change what you are getting in real time. You get data that the exchange sends in the real time feed. The historical data will be cleaned up based on tick corrections (some sent from the exchange, others as determined by our market data group). Thus, the historical data will be cleaner than the data you receive in real time.

---

Jay Froscheiser
DTN - Trading Markets

I'm thinking that might be a problem. If the historical data is 'cleaned up', it doesn't reflect what I might see in real time. So if I test my strategies on historical data, and can make them robust on 'noisy' raw data, the strategies will then work properly during real time, where noise happens in real time. Am I being concerned about nothing?

For the people who use the historical data, and are looking for clean data, do they trade real time? Do they realize the need for testing and trading robust real time algorithms?

Obviously, my vote is for leaving the data alone. If I get junk realtime, I'd like to see junk in my historical data just so I can keep in touch with reality and program accordingly.

Does that make sense?

Makes sense to me. I have the same concern.

In general, it's a good thing for historical data to precisely match what was received in real time -- the tick values, the order in which they appear, etc.

the data on bnzdusd .teurjpy and bgbpjpy has not beeen cleaned up.it is a real pain deleting the data on these hstorical charts.the other majors seem ok.thanks

jay
looking at the historical charts on quotetracker.say bgbpusd.the range is still wrong.perhaps iq could buy in some historical data from a reputable source and use that for historical charting on qt.also the back fill period only goes back 3 years.we really need more.thanks

when you say "the range is still wrong", can you elaborate/further define?

---

Jay Froscheiser
DTN - Trading Markets

the daily range still is not right as far as i can see.i am using point and figure charts on qt which only rcord either the high or the low
compare the candlestick charts with the charts on netdania/daily.the bar range does not seem to be even close as what is on the daily at netdania.i realise this is a different quote source.but in spot fx the quotes are usually onl 1-2 pips away from each other.
trye the daily candlesticks on bgbpusd and compare it with gbpusd on netdania.com