Published on October 18, 2010 11:00 PM

Digital Camera Reviews: Some Criteria Explained

Some of our test criteria aren't particularly easy to explain, whereas others are obvious, and you can spot them as soon as you look at the tech specs. Here are some of the basic criteria we use to assess the quality of a digital camera. It could also be seen as a wish-list of spec and features—manufacturers take note!

Our digital camera reviews are based on four very broad areas of assessment. The first area—handling—is purely practical, covering the design, grip, comfort etc. of the camera. The second is purely technical, as our responsiveness criteria assess factors such as start-up time, photo-to-photo turnaround time, autofocus speed etc. The last two areas judge the quality of the results: picture quality (sharpness, noise, colours etc.) and video quality (sharpness, smoothness, autofocus etc.).

Tz7 tz10 800isoIt's not always easy to quantify the quality of a photo. For example, some cameras 'smooth' the picture to reduce digital noise, which in turn reduces the overall 'sharpness' of the shot (accuracy of fine details, as you can see in the TZ10 test shot, right). Other cameras choose preserve detail, but the pictures look grainy  as a result (as you can see in the TZ7 test shot, right). So which is better? That all depends how you plan on using your camera (it's better to keep detail if you want to post-edit, but it's better to get rid of noise if you want to print pictures out). It's also a question of taste. Judging quality is therefore a subjective business.

However, some simple criteria allow us to quickly identify what kind of score a camera should get. These are the criteria we'll be outlining today, as we have recently re-evaluated and tightened the criteria used to rate cameras in our compact camera reviews. Publishing these basic criteria should help you understand how our reviews process works, but we also hope it draws manufacturers' attention to certain points of interest. We've seen some major gaffes in high-end cameras and we've decided to stop being quite so forgiving as we were in the past.

First of all, the minimum score possible is one star (see DigitalVersus Scoring System). A product with one star is defined as 'a product to be avoided at all costs'.

For the moment, we haven't come across any specific criteria that would stop a camera getting at least two stars. Two stars is the minimum score required for taking photos that are acceptable and usable.

Three StarsNote v3 3

Three stars is the average score and it's by no means a bad score. A camera with three stars does what it's expected to do. It's a product of standard, average quality, which will neither get us wildly excited or fuming with rage.

There are, however, a few criteria that send a camera straight down into the two-star category. These criteria make the camera problematic for daily use.
  • A wide-angle of over 40 mm is pretty useless. You still need to be able to take a full-length photo of someone without having to get them to move back ten metres.
  • No optical or mechanical stabilisation. This function costs pretty much nothing for manufacturers these days, so there's no excuse for it not to feature in a camera. The exception is the handful of fixed wide-angle focal-length models out there (like the Ricoh GR-D) in which it's not essential. On a model costing under £100, we have to admit that the very small extra cost does become a factor, but in such cases, everything else should be good enough to make up for it.
  • Tough 8010 vitesseInfuriating performances. Start-up that takes over three seconds or focusing that takes over two seconds are the minimum acceptable performance levels (the Olympus µ Tough 8010 is so slow it's off the chart, see right).
  • A screen that displays less than 200,000 dots. To take photos you need to be able to line up shots and see what you're snapping with some degree of accuracy. This is the minimum level we consider acceptable.
  • No way of changing the ISO settings manually. These are essential for bypassing the the camera's automatic settings, giving you greater control over noise in the final shot.
  • No HD video. The difference in quality between VGA and 720p HD is pretty big. Therefore, 720p HD is the minimum we'd expect to see these days.
  • Decent image quality at 800 ISO. A 4" x 6" print (10 x 15 cm) or a shot viewed full-screen on a computer shouldn't be too blurred or too full of noise.

Four StarsNote v3 4

For four stars we get a little more demanding. It's no longer about complying with a basic, minimum set of standards to make a camera pain-free to use. For four stars, a camera needs to be good, and good enough to satisfy a relatively demanding user. Here are a few things that could cost a camera its fourth star:
  • A wide angle over 28 mm (essential for landscape shots) or no stabilisation, apart from in the few models with a fixed focal length.
  • No flash off option in auto mode. This is very rare but it does happen! Any consumer camera should definitely have an auto mode, and this mode should allow you to switch the flash off as and when required. Flashes are banned in some museums, for example, and can be annoying in some situations.
  • A screen with tight viewing angles or a low definition (minimum: 320,000 dots). You should be able to line up shots without having to face the screen directly and you should always be able to see your subjects properly!
  • No automatic image rotation. It doesn't really cost anything for manufacturers to include this function and it's frustrating when you have to manually rotate your pictures to get them all the right way round.
  • No zoom in video mode. Although going crazy with the zoom in video mode isn't always a good idea, users should still have the choice.
  • Battery life of less than 200 photos. Even for occasional snappers, that'll get used up very quickly.

Five StarsNote v3 5

For five stars, everything has got to be prefect. For a camera to get our top score, it needs not only to have good all-round performances, but should also have no weaknesses. Make-or-break criteria for a five-star camera are vast and varied. Here are a few factors that could cost a camera its fifth star:
Carte micro sd doigts
Micro-SD cards can be easy to lose.
  • Exotic breeds of memory card. SD cards are affordable, widely available, offer decent performances and are compact. So why would anyone want to use any other card format? We're particularly aiming this gripe at cameras that only take micro-SD cards, which are so small they're easy to lose.
  • Insufficient battery life. A minimum of 300 photos is required for one day of holiday snapping.
  • Mono sound in video mode. A good video with poor-quality sound is ultimately just a bad video.
  • Taking over 2 seconds to start up or to turn around between photos. You should be able to use a five-star camera right at the exact moment you want to, without having to wait for it to sort itself out.
  • Slow focusing. In good light, a camera shouldn't take over 0.7 seconds to focus.
  • Accurate onscreen image. It's important the have an accurate impression of what your photos really look like. For computer monitors, the Delta E (difference between perfect colours and those displayed onscreen) should be under 3 for colours to be considered accurate. The Delta E should be under 8 for a digital camera screen.
  • Decent image quality at 1600 ISO. Shooting indoors is now an everyday occurrence. Shots taken at 1600 ISO should at least be acceptable in quality when viewed full-screen on a computer monitor or when made into 4" x 6" prints (10 x 15 cm).
Obviously, this is a basic list of criteria and it's certainly not exhaustive. Even a camera that ticks all these boxes won't necessarily get five stars, as factors such as the picture quality, design and handling, and loads of other basic details can always tip the balance. On more than one occasion, a camera that's looked 'perfect' on paper has turned out to be incredibly irritating to use due to the smallest of design flaws.

So, manufacturers take note! These criteria are also aimed at you. You now have a check-list of our minimum standards, so you can't say you didn't know a TN screen or a battery life of 150 photos would be a problem in a superzoom compact priced at £300!

Add to favorites