Camera Theory

Introduction

Two operators stand side by side on a show with the same camera and the same lens. One of them can tell you why the background went soft when the director asked for a tighter shot from further back, why the sky turned into stripes on the stream, and why the picture got noisy when they added gain to rescue a dark stage. The other can only tell you that it happened.

This book is for becoming the first operator. It is the theory underneath the camera: what exposure actually is, what a lens does to light, why depth of field behaves the way it does, how a sensor turns light into numbers, what gamma, log and HDR do to those numbers, how colour is carried, and how time enters every picture.

None of it is taught from memory. The sensor chapter follows the industry standard that defines how image sensors are measured. The signal chapters follow the international standards for HD and HDR television. The lens chapters follow two technical papers from Zeiss's camera lens division. Where the book gives a number, it is worked from those sources, and you can work it yourself.

Theory pays on the job in a specific way. It lets you predict. Instead of trying settings until the picture looks right, you know before you touch anything which way it will move and roughly how far.

How to Use This Book

Read it in order. Each chapter uses the one before: exposure is light, the lens controls the light, the sensor counts it, time limits how long it counts, the signal reshapes the count, colour splits it three ways, and the last chapter puts the whole chain together.

Keep a calculator nearby. The book gives a handful of simple relationships and works examples with each, and the exercises ask you to do the same. None of it needs more than multiplication, division and a square root.

Keep a notebook

When something unexpected happens to a picture, write down what you saw, the settings, and the conditions. Then, after the relevant chapter, write down why it happened. After a dozen entries you will find you can predict most of them, which is the point of theory.

A note on the words

"Exposure" here means the amount of light the sensor collects, not the brightness of the picture on a monitor, which also depends on gain and the signal curve. A "stop" is a doubling or halving of light. "Luma" is the brightness signal in video. "Gain" and "ISO" both describe amplification after the light is collected. The standards quoted are international, and their own figures and names are given exactly as published.

A note on safety

A lens concentrates light. That is its job, and it is also why a lens pointed at the sun, or at a laser, can destroy a sensor in moments and can burn an eye at the viewfinder. Never point a camera at the sun without a filter made for it, never look through an optical viewfinder toward the sun, and treat show lasers as hazards to every camera in the room.

The same physics applies to people. The eye is a lens and a sensor too. Anything this book says about concentrated light damaging a camera applies to the eye first.

Light and Exposure

Exposure is not a setting. It is an amount of light, and every camera control either changes that amount or only changes how it is shown.

A camera measures light. Everything else follows from how much arrives.

What exposure is

A camera does not see a scene. It collects light. Every picture element on the sensor gathers the light that falls on it while the shutter is open, and the amount it gathers is the exposure. The sensor standard EMVA 1288 states it plainly: the amount of light a pixel collects depends on the irradiance at the image plane integrated over the exposure time. More light, or the same light for longer, means more signal.

Three things set how much light arrives at the sensor: how bright the scene is, how much of that light the lens lets through, and how long the sensor is allowed to collect it. Gain, or ISO, is not one of them. Gain amplifies the signal after the light has been collected, which is why it brings noise up with the picture.

Stops

Camera people count light in stops. One stop is a doubling or a halving of the light. Opening the iris by one stop doubles the light reaching the sensor; doubling the exposure time does the same; so does doubling the light on the subject.

Light, aperture, time, and gain

The sensor standard EMVA 1288 starts from a simple model: the light a pixel collects is the irradiance on the sensor, integrated over the exposure time. In plain terms, exposure is how bright the image on the sensor is, multiplied by how long the sensor collects it.

That gives three controls that genuinely change exposure. The scene's light, which is the lighting designer's to change. The lens aperture, which sets how much of that light reaches the sensor. And the exposure time, set by the shutter. A neutral density filter belongs with the aperture: it reduces the light through the lens without changing anything else about it.

Gain is different. The standard's model puts amplification after the light has been turned into electrons. Gain multiplies the signal and the noise that came with it. It makes a dark picture brighter on the monitor, and it does not collect one more photon. That is why gain makes pictures noisier, and why an operator who can open the iris or slow the shutter should do that first.

Doubling and halving

Because each of the three controls can halve or double the light, they can be traded against each other in stops. Close the iris one stop and double the exposure time, and the sensor collects the same light as before. Add a two-stop neutral density filter and open the iris two stops, and the exposure is unchanged while the depth of field gets shallower.

Stops add, because doubling is multiplication. Three stops is two times two times two, a factor of eight. Five stops is a factor of thirty-two. Ten stops is a factor of just over a thousand. A scene with ten stops between its darkest important shadow and its brightest highlight has a thousand times more light in one than the other.

It helps to know what a stop looks like. One stop under, the picture is plainly darker but everything is still there. Two stops under, the shadows sink into a murky, noisy grey and the face looks heavy. One stop over, skin starts to lose its modelling and pale shirts lose their folds. Two stops over, the brightest parts have gone flat white, and no correction later can bring back detail that was never recorded. A camera forgives a little under far more readily than a little over.

Neutral density filters

A neutral density filter, ND for short, is grey glass that cuts every colour of light by the same amount. It changes nothing in the picture except its brightness. That is exactly why it is useful: the iris and the shutter both have side effects, on depth and on motion, and ND lets you choose the iris and the shutter for the look you want and then bring the exposure back to where it belongs.

Filters are labelled in two ways, and both count stops. Optical density is a logarithm, and every 0.3 of density is one stop, because 0.3 is the logarithm of two. The other label gives the factor directly: an ND8 lets through one eighth of the light, which is three stops.

Most broadcast and cinema cameras carry ND inside, on a wheel or as an electronic filter just behind the lens mount. A common broadcast set is clear, 1/4, 1/16 and 1/64, which is two, four and six stops. Cinema crews add more in front of the lens, in a matte box.

Daylight shows why so much is needed. Photographers' old rule of thumb, the sunny sixteen rule, says that in full sun the exposure is f/16 with a shutter time of one over the sensitivity. A camera at a sensitivity of 800 would want f/16 at 1/800 second. Video runs at 1/50, which is four stops more light. Opening to f/2 for a soft background is six stops more again. Ten stops too much light: an ND 3.0. Without it the picture is a white sheet; with it, the same sunny lawn looks as it should and the background melts away.

ND has side effects of its own, and they grow with strength. Many filters cut visible light more than the near infrared that sensors can still see, so behind strong ND the share of infrared rises, and black synthetic fabric, dark suits and foliage turn a reddish brown. Filters sold as IR ND cut the infrared as well. A variable ND is two polarising layers turned against each other; at its darkest end it can print a dark cross over the frame, and on a wide lens it darkens the sky unevenly. Cheap filters can tint the whole picture, and every extra piece of glass in front of the lens is another chance for flare.

Why the eye counts this way too

The reason camera people count in stops rather than in units of light is that perception works on ratios. Zeiss's paper on reading MTF curves makes the point in passing: the eye's perception of brightness follows a logarithmic scale, which is why photographers express bright-dark differences in aperture stops.

This is also why video signals are curved before they are stored, as the chapter on the signal explains. If the signal counted light in equal steps, almost all of its steps would be spent on bright tones the eye can barely tell apart, and the shadows would get only a handful.

The reason for counting this way is that the eye judges brightness by ratios rather than differences. A step from one to two units of light looks about as large as a step from fifty to a hundred. Stops turn those ratios into simple counting, so that a lighting change, an iris change and a shutter change can be traded against each other in the same unit.

How to do it

1. Decide which control you want to change, whether light, aperture, exposure time or ND.
2. Count the change in stops, remembering that each stop is a factor of two.
3. If you need to keep exposure the same, change another control the same number of stops the other way.
4. Use gain only after the light-gathering controls are exhausted, and expect more noise.
5. Check the result on the waveform, not the monitor, as Camera Basics teaches.

What goes wrong

Practice it

A camera is at f/4 with an exposure of 1/50 second and no ND. The director wants a shallower depth of field.

The Lens

A lens has one focal length and one maximum aperture. Everything else people say about lenses comes from those two numbers and the size of the sensor behind them.

The lens decides what the sensor sees and how much light it gets to see it with.

Focal length and angle of view

Focal length is a property of the lens. Angle of view is what that focal length does on a particular sensor. A longer focal length magnifies a smaller part of the scene; a shorter one takes in more. On a smaller sensor the same lens shows a narrower angle, which is where the familiar crop factor comes from.

Zeiss's paper on depth of field makes the point that the lens itself does not change when it goes onto a smaller sensor. There is no change of focal length. The smaller sensor simply records less of the image the lens makes.

The angle of view follows from simple geometry: half the sensor's width, divided by the focal length, gives the tangent of half the angle. On a full-frame sensor, 36 mm wide, the familiar focal lengths work out like this.

For longer lenses the rule is nearly exact: double the focal length and the width of the view halves, so everything in the frame becomes twice as large. A large sports box lens on a 2/3-inch camera can run from about 9 mm, a 56 degree view, to about 900 mm, a view of just over half a degree. From the same spot, that lens goes from a whole arena to a single face.

Perspective comes from where the camera stands

Long lenses are said to flatten faces and pull backgrounds forward, and wide lenses to stretch them. The lens is not doing it. How large things look against each other depends only on how far each is from the camera. Change the lens without moving and you get the same perspective, cropped. Move, and the perspective changes whatever lens you then fit to keep the framing.

A face shows it best. The tip of the nose is roughly ten centimetres nearer the camera than the ears. Shoot a close-up from half a metre, and the nose is at 0.4 metres while the ears are at 0.5, so the nose is drawn a quarter larger than it should be against them. Step back to three metres with a longer lens and frame the same close-up: the difference falls to about three percent, which reads as natural. This is why portrait and IMAG close-ups are shot from a distance on a long lens, and why a wide lens pushed into someone's face looks unkind.

The background follows the same rule. At the same framing, the far wall behind a presenter is drawn much larger from a distant camera on a long lens, because the narrow view takes in only a small piece of it. A busy backdrop becomes one or two large, soft shapes. From close in on a wide lens, the same presenter stands in front of the whole room.

The f-number and the entrance pupil

The f-number describes how wide the lens is open. Zeiss defines it on the image side as the ratio of the distance from the image plane to the exit pupil and the diameter of that pupil. On the subject side the useful number is the entrance pupil, the image of the iris seen from the front of the lens, whose diameter is the focal length divided by the f-number.

The same lens on different sensors

Zeiss's depth of field paper is firm that putting a lens on a smaller sensor does not lengthen its focal length. The lens does not know how much of its image the sensor is capturing. A smaller sensor records a smaller part of the same image, so the angle of view is narrower, and the result looks as though the lens were longer.

The ratio between the two sensor diagonals is the crop factor. A typical broadcast camera has a 2/3-inch sensor with a diagonal of about 11 millimetres. A full-frame cinema or photo sensor has a diagonal of about 43 millimetres. The ratio is roughly four, so a 25 mm lens on a 2/3-inch camera shows about the same angle as a 100 mm lens on full frame.

This is why broadcast zoom lenses have short-looking focal lengths and full-frame primes have long-looking ones, and why the same number on two lenses means nothing unless you know the sensor each is meant for.

Pupils, cones and the stop scale

Zeiss defines the f-number from inside the camera, as the ratio between the distance from the sensor to the lens's exit pupil and the diameter of that pupil. What matters for exposure is that this sets the angle of the cone of light reaching the sensor, and so how bright the image on the sensor is. That is why any lens at f/4 delivers the same brightness to the sensor, whatever its focal length.

From the front of the lens, the useful number is the entrance pupil: the image of the iris you see looking into the lens. Its diameter is the focal length divided by the f-number. A 50 mm lens at f/2 has an entrance pupil of 25 mm. A 100 mm lens at f/2 has one of 50 mm. Same f-number, same exposure, twice the pupil, and the larger pupil is what gives the longer lens its softer background, as the next chapter shows.

The f-number scale follows from the pupil. Each full stop shrinks the pupil's diameter by a factor of about 0.71, the reciprocal of the square root of two. That halves its area and halves the light. So the scale goes 1.4, 2, 2.8, 4, 5.6, 8, 11, 16, each number about 1.4 times the last.

Close focus and ramping

The engraved f-number describes the lens focused at a distance. Zeiss notes that at close range the lens no longer has the f-number engraved on the ring, and the effective aperture is smaller, by an amount that depends on the scale and on the lens design. Telephoto designs lose more at close range than symmetrical ones. For live work this is rarely large, but it is why a macro shot can need more light than the iris setting suggests.

Broadcast and cinema lenses add one more effect worth knowing by name. Many zooms cannot keep their widest aperture at the long end of the range, so the picture gets darker as you zoom in past a certain point. Operators call this ramping. It is an optical limit of the design, and the only cure is more light or more gain.

Primes, zooms and box lenses

A prime has one focal length. With fewer moving groups of glass it can be smaller, faster and often sharper than a zoom covering the same view, which is why cinema sets carry matched sets of primes, with their focus and iris rings geared for a follow-focus.

A zoom covers a range, described by its ratio: a 20 times zoom runs from, say, 8 mm to 160 mm. Broadcast zooms are parfocal, meaning focus holds as the zoom moves, so an operator can zoom in tight, focus, and zoom out with everything still sharp. Many stills zooms are not, and lose focus as they zoom, which is fine for a photograph and useless live.

Live production uses two broadcast zoom families. The portable, or ENG, zoom sits on a shoulder camera, with its zoom and focus driven by a hand grip. The box lens is the large rectangular lens on a studio or sports camera, with ratios from about 20 times to over 100 times, driven by demands on the pan bars. Many carry a built-in extender that doubles every focal length. It doubles the f-number too, because the pupil stays the same size while the focal length doubles, so an extender always costs two stops of light.

Cinema lenses are often marked in T-stops instead of f-stops. The f-number is geometry; the T-number is measured, and includes the light lost inside the glass. A lens at f/2 might deliver what an ideal lens would at T2.2. When lenses are swapped mid-shoot, matching T-stops keeps the exposure matching.

Faults you can see in the picture

Every lens bends light imperfectly, and a few of its faults are easy to see once you know their names.

• Vignetting. The corners are darker than the centre, most of all at the widest aperture. Closing a stop or two usually cures most of it, and many cameras correct it electronically.
• Flare and ghosts. A bright light in or just outside the frame bounces between the glass surfaces inside the lens. It shows as streaks, as a chain of coloured shapes in the iris's own outline, and as a grey veil that lifts the blacks. A lens hood or matte box stops the light that is outside the frame.
• Colour fringes. Different colours focus at slightly different points, so a dark edge against a bright background grows a purple or green edge, worst on fast lenses wide open.
• Distortion. Straight lines near the edge of the frame bow outwards on many wide lenses, called barrel distortion, and inwards on some long ones, called pincushion. A truss line or an LED wall edge shows it at once.
• Breathing. On many lenses, turning the focus ring slightly changes the size of the image, so a focus pull looks like a small zoom. Cinema and broadcast lenses are designed to keep it small.

Each full stop on the f-number scale reduces the pupil diameter by a factor of about 0.71, which halves its area and so halves the light. That is why the scale runs 1.4, 2, 2.8, 4, 5.6, 8, 11, 16 and not in round numbers.

How to do it

1. Find the sensor's diagonal in the camera's specification.
2. Divide 43 mm by it to get the crop factor against full frame.
3. Multiply the lens focal length by the crop factor for the full-frame equivalent angle.
4. Divide the focal length by the f-number for the entrance pupil diameter.
5. Check the lens data for ramping at the long end of a zoom.

What goes wrong

Practice it

A 2/3-inch broadcast camera has a sensor diagonal of 11 mm. A full-frame camera has 43 mm.

Depth and Sharpness

Depth of field is not a zone where things are sharp. It is the range where blur is too small to notice, and sharpness fades continuously either side of focus.

Sharpness is not a line in space. It fades gradually either side of focus.

Depth of field

A lens focuses perfectly at one distance only. Points in front of and behind it spread into small blur discs called circles of confusion. Depth of field is the range of distances over which those discs are small enough that nobody notices, which makes it a judgement about viewing conditions as much as a property of the lens.

Zeiss gives the relationships simply. Depth of field grows with the square of the focus distance, shrinks with the square of the focal length, and grows in proportion to the f-number, so closing the iris two stops roughly doubles it. And if you frame the subject the same size, the depth of field barely depends on the focal length at all; what changes is how blurred the distant background becomes.

Sharpness and MTF

Lens makers describe sharpness with the modulation transfer function, or MTF. It measures how much contrast the lens keeps when it images a pattern of fine stripes, from coarse to fine. Coarse detail keeps most of its contrast; fine detail keeps less, until it turns into plain grey.

What depth of field depends on

Zeiss's paper builds the whole subject from the cones of light leaving and entering the lens, and arrives at three rules. If the focus distance doubles, depth of field becomes about four times larger, because it grows with the square of distance. If the focal length halves, at the same distance, depth of field also becomes about four times larger, because it falls with the square of focal length. And depth of field grows in proportion to the f-number, so closing two stops, from f/5.6 to f/11, roughly doubles it.

The paper then adds the rule that surprises operators. If you frame the subject the same size, by stepping back with a longer lens, the depth of field is almost the same as before. The longer lens at twice the distance has twice the entrance pupil, and the two effects cancel. What does change is the distant background: the longer lens blurs it more, because the amount of background blur is set by the physical size of the entrance pupil.

So when a director wants to lose a busy background without losing the presenter's ears, a longer lens from further back does the job, and the reason is the larger pupil, not a shallower depth of field.

Pulling focus between subjects

A lens is truly sharp at one distance only. Turning the focus ring moves that distance nearer or further, and depth of field is the band either side of it where the blur is still too small to notice. Everything else in the frame softens gradually the further it is from that distance, in front as well as behind.

That makes focus a way of directing the eye. Viewers look at whatever is sharp. With three people at three distances and a shallow depth of field, focusing on the nearest, then the middle one, then the furthest tells the audience three different stories from the same frame. Moving focus from one to another during a shot is a focus pull, and it is only possible when the depth of field is shallow enough to separate them. On a small broadcast sensor at a wide angle, all three may be sharp at once, and the pull barely shows.

Every lens also has a minimum focus distance, closer than which nothing can be made sharp. And on lenses that breathe, as the previous chapter explains, the framing shifts a little during the pull.

Bokeh and the iris blades

An out-of-focus point of light, such as a festoon bulb or a distant stage lamp, is drawn as a disc, and the disc is an image of the entrance pupil. Photographers call the look of these discs bokeh, from a Japanese word for blur. With the iris wide open the pupil is round, so the discs are round. Stopped down, the pupil takes the shape of the iris blades, and the discs turn into polygons with as many sides as the lens has blades, as well as becoming smaller.

Near the corners of the frame the discs are often clipped into cat's-eye shapes, because the lens barrel cuts off part of the pupil when seen at an angle. That is the same mechanical cause as vignetting.

Sensor size and equivalent f-numbers

The same reasoning explains why small sensors look so deep. Zeiss works out equivalent f-numbers: for the same angle of view and the same depth of field, the f-number scales with the sensor's size. Its table puts f/2.8 on a 2/3-inch sensor at about f/8 to f/11 on full frame.

That is why a broadcast camera wide open at f/2.8 still shows a stage almost entirely sharp, while a full-frame camera at f/2.8 can isolate one face. It is also why making a small-sensor camera look "cinematic" mostly means long lenses, wide apertures and distance from the background, since the sensor cannot do it alone.

The paper adds a qualification about the f-number used for exposure: converting f-numbers between formats is right for depth of field and wrong for exposure. An f/2.8 lens delivers the same brightness to any sensor.

Hyperfocal distance, and how sharp is sharp

The hyperfocal distance is the closest focus at which the depth of field reaches infinity. Focus there, and everything from half that distance to the horizon is acceptably sharp. Zeiss gives the approximation: the hyperfocal distance is roughly the focal length squared divided by the product of the f-number and the acceptable blur circle. Its own example, an 85 mm lens at f/2 on full frame with a blur circle of 1/1500 of the diagonal, comes to about 129 metres.

"Acceptable" is the weak word in all of this. Zeiss explains that the usual blur circle of 1/1500 of the image diagonal comes from how a print is viewed, and that it corresponds to the eye's ability to resolve about one minute of arc. On a large screen viewed close up, the same blur is plainly visible. For an IMAG screen ten metres wide, or a 4K monitor at arm's length, depth of field tables are optimistic.

Two more points from the paper are worth keeping. The familiar rule that depth of field extends one third in front of the focus and two thirds behind is not generally true; it holds only at one distance for each aperture, and close up the split is nearly even. And stopping down does not help without limit: diffraction spreads every point into a disc whose diameter in micrometres is about the f-number, so at f/16 every point is about 16 micrometres wide whatever the focus.

Reading a lens datasheet

Zeiss's MTF paper explains the curves lens makers publish. The lens is tested with patterns of light and dark stripes of increasing fineness, counted in line pairs per millimetre. For each, the MTF is the contrast the image keeps divided by the contrast of the pattern. Coarse stripes keep most of their contrast; fine stripes keep less.

A data sheet usually shows MTF at 10, 20 and 40 line pairs per millimetre, plotted from the centre of the image to the corner, with two lines each for stripes running toward the centre and across it. The 10 line pair curve speaks to contrast and punch, the 40 line pair curve to fine detail. Curves that stay high and close together to the corner describe a lens that is sharp across the frame.

The paper explains what sits behind those curves. No lens brings a point of light back to a perfect point; it spreads it over a small area, the point spread function, which the paper calls the lens's brushstroke. Contrast is defined as the brightest value minus the darkest, divided by the two added together. The two lines per frequency on a datasheet are for stripes running toward the centre, called sagittal, and across that direction, called tangential, because a lens's brushstroke is often not round away from the centre.

The paper also shows two effects that move sharpness around the frame and in depth. Field curvature means the sharpest surface is not quite flat, so the corners can be best at a slightly different focus from the centre. And focus shift means the best focus moves slightly as the lens is stopped down. Neither usually matters for live work; both explain why a lens tested on a flat chart can behave differently on a deep stage.

Two cautions close the paper. MTF measured in white light hides colour fringes caused by the different colours coming to focus at slightly different distances, and these fringes can appear purple in front of the focus and green behind it on fast lenses. And some published MTF curves are calculated from the design rather than measured from real lenses, which can make them look better than any real lens is.

The paper warns against two shortcuts. A single resolution number is a poor measure, because it depends heavily on the contrast threshold chosen. And MTF says nothing about veiling glare, the wash of stray light that lifts blacks, so a lens with fine MTF curves can still give a flat picture into a bright backlight.

Zeiss's paper on reading MTF curves stresses that the sharpness of the whole picture is the product of every link in the chain: the lens, the filter in front of the sensor, the sensor, the processing, the display and the eye. The weakest link sets the limit, and a superb lens on a soft display looks no better than an ordinary one.

How to do it

1. Decide what must be sharp, from the nearest point to the furthest.
2. For more depth, go wider, step back, or close down, in roughly that order of strength.
3. For a softer background without losing the subject, step back and use a longer lens at the same framing.
4. On a small sensor, expect deep focus and use distance from the background to separate the subject.
5. Judge sharpness on the screen the audience will see, not the viewfinder.

What goes wrong

Practice it

A presenter stands 3 metres from the camera in front of a busy video wall 2 metres behind them.

The Sensor

A sensor counts photons. Two numbers describe it: the most it can hold, and the fewest it can count reliably. Everything else follows.

A sensor counts photons. Its quality is how few it can count reliably and how many it can hold.

From photons to numbers

EMVA 1288 describes the sensor as a chain. Photons hit the pixel during the exposure. A fraction of them, the quantum efficiency, frees electrons. The electrons form a charge, which is converted to a voltage, amplified, and digitised into a number. In a linear sensor the number rises in proportion to the light.

Each pixel can hold only so much charge, and the digital range ends at the highest number the converter can produce. Beyond that point the pixel is saturated and the detail is gone. No amount of later adjustment brings it back, which is why camera people protect highlights.

Noise and dynamic range

Light itself is noisy. Photons arrive at random, so the count in a pixel fluctuates even under perfectly steady light. EMVA 1288 calls this shot noise and notes that it is given by the basic laws of physics and is the same for every camera. On top of it come the noise of the read-out electronics and the rounding of the converter.

The chain inside a pixel

EMVA 1288, the standard used across the industry to measure image sensors, describes what happens in each pixel as a chain. Photons hit the pixel during the exposure. A fraction of them, the quantum efficiency, frees electrons. The standard notes that its quantum efficiency refers to the whole pixel area, so it includes the fill factor and the effect of the tiny microlenses that steer light onto the sensitive part. The electrons are converted to a voltage, amplified, and digitised.

In a linear sensor, which is what the standard assumes, the digital number rises in proportion to the photons collected, until the pixel or the converter runs out of room. That is the raw material. Every curve, colour and look the camera produces is applied to these numbers afterwards.

Why noise is a law of nature

Even under perfectly steady light, the number of photons arriving in a pixel varies from one frame to the next, because light comes in discrete particles arriving at random. The standard calls the resulting fluctuation shot noise, and states that it is given by the basic laws of physics and is equal for all types of camera. Its size follows a simple rule: the variance equals the mean, so the noise is the square root of the signal.

Work it through. A pixel that collects 10,000 electrons has shot noise of about 100 electrons, a signal-to-noise ratio of 100. A pixel that collects only 100 electrons has noise of about 10, a ratio of 10. Four times the light doubles the signal-to-noise ratio. That is why shadows are always noisier than highlights, on every camera ever made.

The standard adds two more noise sources on top: the noise of the read-out electronics, which is the same at every light level, and the rounding of the converter. In the dark, these dominate. In good light, shot noise does. The standard's ideal sensor, with no read-out noise at all, would still have shot noise.

Gain does not change this arithmetic. It multiplies signal and noise together. The only way to improve the ratio is to collect more light.

The top and bottom of the range

At the top, a pixel saturates. The standard is careful to distinguish two limits: the full-well capacity, which is how much charge the pixel can physically hold, and the saturation capacity, which is usually lower because the converter clips the signal at its highest number before the pixel is full. Either way, light beyond that point is recorded as the same maximum value, and the detail is gone.

At the bottom, the standard defines the absolute sensitivity threshold as the light needed for a signal-to-noise ratio of one, the point where signal equals noise.

Dynamic range is the ratio between the two: saturation divided by the sensitivity threshold. Expressed in stops, it is the span between the brightest highlight the sensor can hold and the darkest shadow it can still tell from its own noise. As an illustration, a pixel that saturates at 30,000 electrons with a threshold of 3 electrons has a ratio of 10,000 to one, a little over 13 stops.

Heat, patterns and dead pixels

Electrons are also freed by heat, with no light at all. The standard calls this dark current, notes that it is mainly caused by thermally induced electrons, and that it increases with temperature. Older sensors roughly doubled their dark current over a fixed rise in temperature; the current release of the standard no longer assumes a single model, because many modern sensors behave differently. A camera that has been running in a hot truck, or shooting long exposures, shows more noise in its blacks than the same camera cool.

Not every pixel is identical either. The standard describes two basic differences: in the dark level from pixel to pixel, and in sensitivity. It singles out periodic patterns, especially rows and columns, as the most damaging, because the human eye detects them very sensitively. Faint vertical or horizontal lines in a dark, gained-up picture are this effect.

The standard also describes gradual variations across the whole chip, and notes that the eye does not detect them at all and that lenses add more of the same, in the form of shading and vignetting, the darkening toward the corners. Unlike row and column patterns, gradual shading is rarely a problem in a picture. It matters when cameras must match or a flat field must look flat, and it is usually corrected for the lens and camera together.

And some pixels are simply outliers: stuck bright, stuck dark, or far more sensitive than their neighbours. Cameras map and hide them, and many let an engineer rerun that mapping. A sudden bright speck that stays in the same place on every shot is usually one of these.

The standard defines dynamic range as the ratio between the light that saturates the sensor and the smallest amount it can detect against its own noise. It is the span, in stops, between the brightest highlight a camera can hold and the darkest shadow it can still separate from noise.

How to do it

1. Give the sensor as much light as the shot allows before adding gain.
2. Protect highlights, since anything past saturation is lost for good.
3. Expect shadows to be noisier than highlights, and light for it.
4. Keep cameras cool, especially for long or gained-up work.
5. Rerun the camera's pixel mapping if a fixed speck appears.

What goes wrong

Practice it

Two pixels on the same sensor: one in a highlight collects 16,000 electrons, one in a shadow collects 400.

Time

Every picture is an exposure over time. How long, and how often, decide how motion looks and whether lights flicker.

Frame rates, and the odd ones

BT.709, the HD standard, lists the picture rates in use: 24, 25, 30, 50 and 60 frames per second, and for 24, 30 and 60 it also specifies the same rates divided by 1.001. Those give the familiar odd numbers: about 23.976, 29.97 and 59.94. They exist for compatibility with the colour television systems of North America and Japan, and they are why so many cameras offer both "24" and "23.98".

The same standard fixes the HD picture itself: 1920 samples across by 1080 lines, square pixels, a 16:9 shape. Pictures can be captured progressively, a whole frame at a time, or interlaced, as two fields of alternate lines captured at different moments. A third option, the segmented frame, carries a progressively captured picture as two halves, odd lines then even, so it can travel through equipment built for interlace. The picture is still progressive; only the transport is split.

The standard's attachment on film describes two conversions worth knowing. Material shot at 24 frames can be played at 25 by running it slightly fast, with no loss of picture quality, which is how film reaches 50 Hz countries. And it can be converted to 30 by inserting a 3:2 pulldown, repeating fields in a pattern, which is how it reaches 60 Hz ones.

Shutter angle and motion blur

The exposure time per frame sets how much a moving object smears. Film cameras described it as a shutter angle, because their shutter was a spinning disc with an opening, and the term has stayed. The relationship is simple: the shutter angle is the exposure time multiplied by the frame rate, multiplied by 360 degrees.

At 25 frames per second, an exposure of 1/50 second is half the frame time, a 180-degree shutter. At 24 frames, 1/48 second is also 180 degrees. That half-frame exposure gives the motion blur audiences associate with film and drama. A much shorter exposure freezes each frame and makes fast motion look staccato; a much longer one smears it.

Live television mostly keeps the exposure near the frame time, and sport often shortens it so that replays and slow motion stay crisp. Every choice here also changes exposure, as the first chapter showed: halving the exposure time costs a stop.

Rolling shutters

Many sensors do not capture the whole frame at one instant. They read it out row by row from top to bottom, so the top of the picture is captured slightly earlier than the bottom. This is called a rolling shutter. It is invisible on a still subject and shows itself in three ways when things move.

• Skew. A fast pan leans vertical lines, because each row saw the scene a moment later than the one above.
• Wobble. Vibration makes the picture ripple, because different rows see the camera in different positions.
• Partial flash. A camera flash or a strobe that lasts less than the read-out lights only part of the frame, leaving a bright band.

Sensors that capture every row at once are called global shutter sensors, and avoid all three. For an operator the practical response is to know which kind of camera you have, and to be gentle with fast pans on a rolling-shutter one.

Flicker, and why exposure time cures it

Many light sources flicker, too fast for the eye to see but not too fast for a camera. Many fluorescent, discharge and LED fixtures run from mains power and pulse at twice its frequency: 100 times a second in 50 Hz countries, 120 times a second in 60 Hz ones.

In a venue that usually means the house lights, the room's own ceiling and wall fixtures, which nobody on the AV crew controls and which often stay on for conferences and meetings. Stage lighting on dimmers or LED fixtures may flicker differently again, so check every kind of light that falls on the subject.

If each exposure lasts a whole number of flicker cycles, every frame collects the same amount of light and the flicker disappears. Under 120 Hz flicker that means 1/60 or 1/120 second; under 100 Hz flicker, 1/50 or 1/100. Shoot 1/50 under 60 Hz lighting, and each frame catches a slightly different part of the cycle, which appears as a slow brightness pulse or a band rolling through the picture.

LED video walls have their own refresh rates and scanning patterns, and Camera Basics covers what to do about them. The principle is the same: the camera's exposure has to sit comfortably over the light's own timing.

How to do it

1. Use the project's agreed frame rate, including whether it is 1000/1001.
2. Start from an exposure of about half the frame time.
3. Shorten the exposure for crisp sport and replay, and make up the lost stops with light or aperture.
4. Under mains lighting, choose an exposure that is a whole number of flicker cycles.
5. On a rolling-shutter camera, avoid whip pans and strobes where you can.

What goes wrong

Practice it

A conference in a 60 Hz country, recorded at 29.97 frames per second, under LED house lights that run from mains power.

From Light to Signal

The sensor's numbers are reshaped by a curve before they leave the camera, and the curve decides what correct exposure looks like.

The numbers from the sensor are reshaped before they ever leave the camera.

Gamma and transfer functions

A sensor responds in proportion to light, but people do not see that way, and a video signal with too few steps would waste most of them on highlights the eye barely distinguishes. So cameras apply a curve, the opto-electronic transfer function, before the signal leaves. The HD standard, ITU-R BT.709, specifies it: a power law of about 0.45 over most of the range, with a short straight section near black.

The display applies the opposite curve to turn the signal back into light. The HDR standard, ITU-R BT.2100, names all three parts: the camera's curve, the display's curve, and the overall relationship between scene light and screen light, which it calls the rendering intent.

Levels, bit depth and HDR

BT.709 fixes where black and white sit: 16 and 235 in 8-bit, 64 and 940 in 10-bit. Between black and white that gives 219 steps in 8-bit and 876 in 10-bit, four times as many. The codes above white and below black are headroom: room for overshoots, highlights a little above nominal white, and test signals, which is why a waveform can legitimately show signal above 100 per cent. A smooth gradient such as a blue sky across a wide shot can show visible bands when it has to be drawn with too few steps, especially after grading stretches it.

BT.2100 defines two ways to carry high dynamic range. PQ, Perceptual Quantization, ties each signal value to an absolute brightness on the display, up to 10,000 candelas per square metre, on a curve tuned to human vision. HLG, Hybrid Log-Gamma, is relative, uses a curve close to conventional television at the bottom and a log curve at the top, and offers a degree of compatibility with ordinary displays. The standard says a production should use one or the other consistently and not mix them.

Two more figures from the standard are useful on set. HDR reference white, the level of a white card properly exposed, is 203 candelas per square metre, far below the peak, which leaves room for real highlights. And the standard advises producing at the highest practical resolution and scaling down for distribution, because that gives better quality than producing at the distribution resolution.

Two settings in the standard often trip up people moving between systems. Narrow range, where 10-bit black is 64 and nominal white 940, is the default for production and exchange. Full range, where the whole span from 0 to 1023 is used, is newly defined and, the standard says, should not be used for exchange unless all parties agree. A picture that looks washed out or crushed when it arrives somewhere else has often been read in the wrong range. And in HLG the display applies a system gamma of 1.2 at a peak of 1,000 candelas per square metre, adjusted by formula for brighter or dimmer displays.

The HD curve, worked through

BT.709 gives the camera's transfer function exactly. For most of the range the signal is 1.099 times the light raised to the power 0.45, minus 0.099. Below a very low light level, 0.018 of white, the curve becomes a straight line, 4.5 times the light, to avoid amplifying noise near black.

Put a familiar value through it. A grey card reflecting 18 percent of the light that reaches white has a light value of 0.18. The curve turns that into a signal of about 0.41. On a 10-bit waveform, where black is 64 and white is 940, that lands at about 422, roughly 41 percent of the way up. That is why middle grey does not sit at 18 percent on a waveform.

The curve spends its steps where the eye needs them. Light values from zero to 0.18 take up about 41 percent of the signal; the brightest half of the light, from 0.5 to 1, gets under a third, about 29 percent. The standard notes one more thing worth knowing: in practice the camera's curve is adjusted so that the picture has the desired look on a reference monitor. The formula is the starting point, and every camera's knee, black gamma and detail settings move away from it.

Camera curve, display curve, and the space between

The HDR standard, BT.2100, sets out three functions. The opto-electronic transfer function, or OETF, turns scene light into signal in the camera. The electro-optical transfer function, or EOTF, turns signal into light in the display. And the opto-optical transfer function, or OOTF, is the overall relationship between the light in the scene and the light on the screen, which the standard calls the rendering intent.

The standard explains why the overall result is not a straight copy of the scene: the reference rendering compensates for the difference in how we perceive tone in the bright scene and in the dimmer room where the screen is watched. It also notes that only two of the three are independent; given any two, the third follows.

Log curves

Camera makers add their own curves for recording, called log curves, such as Sony's S-Log3, ARRI's LogC and Canon's Log 3. Each one spreads the sensor's full dynamic range evenly across the signal, stop by stop, so that the recording holds as many stops as the sensor can see. The picture looks flat and grey until a lookup table, or LUT, converts it back for viewing.

For exposure this matters directly. On a log recording, middle grey sits at the level that maker's curve specifies, not where BT.709 puts it, and the gap above it is what holds the highlights. Exposing a log picture to look "right" on an uncorrected monitor usually means overexposing it badly. Expose to the maker's figures, and judge the look through the LUT.

How to do it

1. Find out which curve the camera is recording, whether BT.709, a log curve, PQ or HLG.
2. Look up where middle grey and reference white should sit for that curve.
3. Expose a grey card to that level on the waveform.
4. Judge the look through the correct LUT, not on an uncorrected monitor.
5. Record in 10-bit or more when the picture will be graded or keyed.
6. In HDR, keep one system, PQ or HLG, throughout the production.

What goes wrong

Practice it

Use the BT.709 curve, with 10-bit black at 64 and white at 940.

Colour

A camera does not see colour. It sees three filtered amounts of light and builds every colour from them, against a white it has been told about.

A camera sees three colours and builds every other one from them.

Primaries, white and luma

Video describes every colour as a mix of red, green and blue primaries, and a standard fixes exactly which red, green and blue. BT.709 specifies its primaries and a white point called D65; BT.2100 specifies much purer primaries for a wider range of colours.

Brightness in video, called luma, is a weighted sum of the three. BT.709 gives the weights: about 71 percent green, 21 percent red and 7 percent blue. Green carries most of what the eye reads as detail and brightness, which is why green is the colour that most affects exposure and focus.

Colour sampling

Because the eye is less sensitive to fine colour detail than to fine brightness detail, video often stores colour at lower resolution than brightness. BT.2100 defines the three common patterns: 4:4:4, where colour has full resolution; 4:2:2, where it is halved horizontally; and 4:2:0, where it is halved horizontally and vertically.

How a single sensor sees three colours

Most cameras with one sensor put a mosaic of tiny colour filters over it, so that each pixel measures only red, green or blue light. The common pattern has twice as many green filters as red or blue, matching the eye's greater sensitivity to green. The camera then estimates the two missing colours at every pixel from its neighbours, a step called demosaicing or debayering.

EMVA 1288 mentions debayering in passing, as one of the processing steps that break the simple linear model of a sensor, alongside noise reduction and edge sharpening. That is the useful point for an operator: by the time a picture leaves the camera, colour has been estimated and the image has been processed, and the finest detail, especially in strongly coloured areas, is partly reconstruction.

Broadcast cameras with three sensors avoid the estimate. A prism splits the light into red, green and blue, and each colour has a whole sensor to itself.

Gamut and the white point

A standard fixes exactly which red, green and blue a signal means. BT.709 gives its three primaries as coordinates on the international colour chart. BT.2100 specifies much purer primaries, each defined as a single wavelength of light: red at 630 nanometres, green at 532 and blue at 467. The colours a system can show lie inside the triangle its primaries make, and the BT.2100 triangle is far larger, which is what wide colour gamut means.

Both standards use the same white, called D65, a standard daylight with fixed coordinates. Equal red, green and blue signals mean that white. White balance is the camera's way of meeting it: the operator points the camera at something neutral under the light of the scene, and the camera scales its red, green and blue gains until that object produces equal signals. Warm tungsten light needs the blue lifted and the red lowered; cool daylight needs the opposite.

Kelvin, and what the wrong balance looks like

The colour of a light source is described by its colour temperature, in kelvin: the temperature at which a glowing body would give off light of that colour. The scale runs the opposite way to how it feels. Low numbers are orange, high numbers are blue.

When the camera is balanced to the light, a white shirt is white. Balanced for 5,600 K under 3,200 K tungsten, the camera is ready for bluer light than it gets, and the whole picture turns orange. Balanced for 3,200 K in daylight, it turns blue. Skin shows the error first, because viewers know what skin should look like.

A camera has one balance for the whole frame. When a scene mixes sources, such as blue daylight through a window and orange tungsten inside, no setting can make both neutral: balance for one and the other goes orange or blue. The cure is on the lights, with orange or blue gel to bring them together, not in the camera.

Colour temperature is not the whole story. Daylight and tungsten give off a smooth spread of every colour. Many LED and fluorescent sources give off spikes and gaps, and a camera sees those gaps differently from the eye. Skin can go green, sallow or grey under a light that looks fine in the room. Balancing also has a second axis, green to magenta, for exactly this. The EBU publishes a television lighting consistency index, TLCI, which scores a light by how well a camera reproduces colours under it.

Luma, and why green dominates

Video carries brightness separately from colour, as luma. BT.709 defines HD luma as 0.2126 of red, 0.7152 of green and 0.0722 of blue. BT.2100 uses slightly different weights for its wider primaries, 0.2627, 0.6780 and 0.0593. In both, green carries most of the brightness.

That has practical consequences. A fully saturated blue, at the top of its range, contributes only about 7 percent of full brightness to the luma signal, so blue light on a stage can look bright to the eye and read dark on a waveform. Focus aids and sharpness in video lean on green for the same reason. And a picture that looks correctly exposed in green can still clip in red or blue, which is why some waveforms can show the three colours separately.

What subsampling costs a key

Because the eye resolves fine brightness detail better than fine colour detail, video often stores colour at lower resolution. BT.2100 defines the three common patterns: 4:4:4, where colour has the same resolution as luma; 4:2:2, where colour is halved horizontally; and 4:2:0, where colour is halved horizontally and vertically. BT.709's own colour-difference signals are sampled at half the horizontal rate of luma, which is 4:2:2.

For ordinary viewing the saving is invisible. It shows when a picture is keyed or heavily graded. A chroma key has to decide, pixel by pixel, what is green screen and what is subject, and with colour at half or a quarter of the resolution, the edge of a sleeve or a strand of hair becomes blocky. That is why keying and grading prefer 4:2:2 or 4:4:4 recordings.

The saving is real and usually invisible, until the picture is keyed or heavily graded. Then the coarser colour shows at sharp coloured edges, which is why green-screen work and grading prefer 4:2:2 or better.

How to do it

1. White balance on a neutral white or grey under the scene's actual light.
2. Rebalance whenever the light source changes.
3. Check saturated colours, especially blue and red, for clipping on a per-colour waveform.
4. Record 4:2:2 or better for anything that will be keyed or graded.
5. Agree the colour standard, BT.709 or BT.2100, for the whole production.

What goes wrong

Practice it

Use the BT.709 luma weights of 0.2126 red, 0.7152 green and 0.0722 blue.

The Whole Chain

A picture passes through a dozen stages between the scene and the eye. The quality the audience sees is set by the weakest of them.

Sharpness multiplies

Zeiss's MTF paper states the rule that ties this book together: the MTF of the entire imaging chain is approximately the product of all the individual MTFs. Each stage keeps a fraction of the contrast in fine detail, and the fractions multiply. A lens that keeps 80 percent at some fineness, a sensor that keeps 60 percent, and a display that keeps 70 percent leave about 34 percent at the end.

The paper's corollary is that the product is always smaller than the smallest factor. The weakest link sets the limit, and improving any other link barely helps. A superb lens on a camera with heavy compression, or feeding a soft projector, is wasted.

The links, in order

Every chapter in this book described one link. Laid end to end they are the path every picture takes.

Two links need a word here. Zeiss notes that most sensors have an optical low-pass filter in front of them, made of birefringent plates, which deliberately enlarges every point of light to suppress moire. It is the camera blurring its own picture on purpose, because the alternative is false patterns in fine detail. And the paper explains that the sharpening applied in processing, which brightens one side of an edge and darkens the other, raises how sharp a picture looks without adding any detail. Its conclusion is that apparent sharpness and resolution are not the same thing.

Viewing distance closes the chain

The last link is the viewer. BT.709 defines an HD system as one designed for viewing at about three times the picture height, where a discerning viewer with normal sight sees nearly the quality of the original scene. BT.2100's reference viewing environment puts 1080-line pictures at 3.2 picture heights and 4K pictures at between 1.6 and 3.2.

Zeiss's depth of field paper gives the reason underneath: the eye resolves detail down to about one minute of arc, so what looks sharp depends on how large the picture appears to the viewer. For AV this is the IMAG problem. A camera picture sized for a monitor three picture heights away is shown on a screen the front row sees from much closer in terms of picture heights, and every softness, noise grain and focus error in the chain becomes visible.

Glossary

How to do it

1. Describe the fault exactly: soft everywhere, soft in one plane, noisy, banded, smeared in motion, or blocky in colour.
2. Match it to the link in the table that causes that kind of loss.
3. Check that link first, before changing anything else.
4. Remember sharpening hides softness without adding detail.
5. Judge the result at the audience's viewing distance, on their screen.

What goes wrong

Practice it

A lens keeps 80 percent of contrast at a given fineness of detail, the sensor and its filter keep 50 percent, and the projector keeps 60 percent.

Where to Go Next

This book covered why pictures behave as they do. Camera Basics and Camera Intermediate cover operating the camera with that knowledge. Video Engineering covers matching cameras and running the video system. Visual Aesthetics for AV covers how a picture should look. And Physics of Sound is this book's counterpart for audio.

Every source behind this book is free to read. The EMVA 1288 standard is published by the European Machine Vision Association. ITU-R BT.709 and BT.2100 are on the ITU's website. And Zeiss's two papers by H. H. Nasse, Depth of Field and Bokeh and How to Read MTF Curves, are among the clearest writing on lenses anywhere.

The first ninety days

Four things. Put a grey card in front of your camera and find where it sits on the waveform in each curve the camera offers. Work out the entrance pupil of every lens you use. Shoot the same framing with a short lens close and a long lens far, and compare the backgrounds. And next time a picture surprises you, find the link in the chain that caused it before you change a setting.

Where this leads

Operators who understand the theory become the people others ask: the camera engineer, the director of photography, the person who can say before the show what will go wrong. The theory does not replace experience. It makes experience add up faster.

What the gear actually looks like

Every device named in this book, in one place. Recognition is the whole point of Chapter 4 — use this page to attach a picture to each name before you meet it on a dock at six in the morning.

Audio

Dynamic microphone

Condenser microphone

Wireless handheld

Radio rack

Mixing console

Stage box / snake

Power amplifier

Main loudspeaker

Subwoofer

Stage monitor wedge

In-ear monitors

DI box

Video

Camera

Video switcher

Scaler / processor

Projector

LED wall panel

Confidence monitor

Lighting & control

Conventional fixture

Moving-head fixture

Lighting console

Dimmer

Show controller

Media server

Networking & infrastructure

Network switch

Patch bay

Equipment rack

Power distribution

Cable tester

A properly coiled cable

Connectors, close up

XLR

TRS / TS ¼-inch

SDI on BNC

HDMI

Cat5e / Cat6 (RJ45)

Fiber optic

DMX (5-pin XLR)

AC power — not signal