Several months ago, we introduced a Data Upload function into FoCal. This allows users to choose to upload their test results to Reikan, which was intended to give us an opportunity to start to look for patterns in camera and lens behaviour, so we could highlight possible issues, make FoCal better and also give the opportunity for you as a user to be able to compare your results against averages of a large amount of data.
We’re not quite there yet with the latter parts, but we thought we’d take a very quick look at the data captured so far to show that we are really intending to do useful things!
15 Most Popular Lenses (Canon and Nikon)
First off, we wanted to show a selection of the most popular lenses used with FoCal (these are from the Fully Automatic test):
I don’t think there are any big surprises there. The 24-70 is a very popular lens length on both manufacturers, and the 24-105 is a popular Canon option (the Nikon 24-120 is certainly not used as much as the Canon 24-105 from the looks of this data). The 70-200’s are also very close to the top, which again is not a surprise as they’re very popular lenses. It’s nice to see that the data captured from FoCal does match the real-world expectations.
Focus Consistency
I’m hesitant to put these graphs up because the data size is not massive for them, so it’s important to be very careful of any conclusions drawn from this data.
First, I should explain what the graphs mean:
- The green (median) bar is the median average of the “Consistency of Focus” results from all the (valid) uploaded tests run by users of FoCal. This gives a measure of the quality, with 100% being absolutely perfect (which no camera/lens ever reaches).
- The red (range) bar is the range of values in the results, and it’s in the same units as the green bar. This gives a measure of how consistent the results are, the smaller the number the more similar the range of results were. There is no filtering on this data so there are likely to be a few rogue values here and they will push the bar size up, so it’s not quite as useful a measure of data quality as it first looks.
- The blue (standard deviation) bar gives a measure of the quality of the data set. Again, a smaller value indicates less spread of data.
Canon
The Canon graph below shows that most lenses achieve >95% consistency, except for 3 of the top 15 – the 50/1.8, the 500/4 and the 70-210. I would forget about the 500mm lens for two reasons – first, the data size will be quite small as it’s an expensive lens (and you’ll note it didn’t show up in the most popular 20 lenses), and also it’s a very long focus length so the test setup needs to be absolutely solid to get consistent and meaningful results with a lens this length.
That leaves the 50/1.8 and the 70-210. Again, the 70-210 is not a particularly popular lens so the data set may be less valid, but it may also be due to the older design on the lens. Roger Cicala over at LensRentals did some very good analysis on the autofocus performance between older and newer lenses and cameras and found significant differences (see http://www.lensrentals.com/blog/2012/07/autofocus-reality-part-ii-1-vs-2-and-old-vs-new). The 50/1.8 has a lot of data associated with it however, and from personal experience in the FoCal lab, we can definitely say that the EF 50mm f/1.8 lens is not the best focusing lens out there! It’s cheap and the motor drive is somewhat inconsistent, so it’s not surprising that there’s a lot of range to the results and a poorer CoF value.
Nikon
The Nikon chart again has a couple of noticeable dips, but both of these are for longer telephoto lenses so again, the data size and test condition variability may well be the cause of these lenses appearing slightly poorer.
Closing Thoughts
As I mentioned above, it’s really important to not draw conclusions from these graphs that aren’t scientifically evident, and with a relatively small (valid) data set the purpose of this post was just to show a little view of what data we have available. What we can say from this data is that over a reasonable test set, there doesn’t appear to be a significant difference between the Canon and Nikon autofocus performance as measured by the FoCal AF Consistency Test.
If you use FoCal, it’s really useful if you can enable the Data Upload feature. No images are uploaded (we don’t have the space!), and nothing that personally identifies you is uploaded other then your camera serial number and that’s purely used to spot patterns in ranges of cameras, and to group sets of data together. If you want, you can even stop your serial number being sent (there’s a tick-box to make the uploaded data anonymous) and it’s still useful to us (although not quite as useful as we can’t group tests together as well!)
There have been a lot of changes to the analysis and data upload functionality of FoCal as it’s developed over time, and whilst quite a few of the early results are not useful for analysis (that’s why we’re working with smaller data sets today), all data uploaded now is very relevant and useful for creating this sort of information to feed back.