How a computer sees

The idea

A computer stores a picture as a grid of brightness numbers, one for each pixel, and finds patterns in them.

What does a computer see?

A phone takes a photo of a diya. What does the computer inside actually "see" in that photo?

• A picture with colours and shapes, just like I see
• A long list of numbers, one for each tiny square
• Nothing, until a person describes the photo to it

Paint with numbers

A picture on a screen is made of tiny squares. Paint at least 10 squares dark by tapping or dragging, and watch the numbers.

Each square shows how bright it is. Here 0 is the darkest and 1 is the brightest.

Be the computer

Now you are the computer. It was given only numbers, no picture. Paint a dark square wherever the number is 0.

A picture is numbers

A picture is a grid of tiny squares called pixels. Each pixel is stored as one number for how bright it is. The computer is given only these numbers.

From 64 numbers, a diya appeared on your grid. A program finds shapes like this by spotting patterns in the numbers. For example, a run of 0s with bright numbers beside it may be a line.

To a computer, a picture is a grid of numbers, one brightness number for each pixel.

Using pictures this way is called computer vision. Real photos have millions of pixels and many shades.

Notes

To a computer, a picture is a grid of numbers, one brightness number for each pixel.

A program finds shapes by looking for patterns in the numbers. Pictures of people are personal data, so ask first.

Check yourself

A tiny black-and-white icon is 16 pixels wide and 16 pixels tall. How many brightness numbers does the computer store for it?

Answer: 256

There are 16 rows, and each row has 16 pixels. So the icon has 16 × 16 = 256 numbers. Colour photos use three numbers per pixel.

A program is searching a picture for a dark straight line. What does it look at?

A school wants a camera that reads students' faces to mark attendance. What should the school do first?

• The shapes, the way our eyes pick them out at a glance. A computer has no eyes. It works only with the pixel numbers, so any shape has to be found inside them.
• The numbers, looking for small ones with big ones beside them — correct. Yes. A dark line is a row of low brightness numbers with higher ones on each side. The program finds it by checking the numbers.
• The file name, which says what the picture shows. A file name can be anything, such as IMG 204. The pattern that shows a line is in the pixel numbers.
• Start now, because the numbers made from a face cannot point to any person. Numbers made from a face can still point to one person. That makes them personal data.
• Install the camera quietly so that nobody feels worried about it. Secrets break trust. People have a right to know when their face is recorded, and why.
• Ask students and parents first, and explain how pictures are kept safe — correct. Yes. Faces are personal, so people should agree first and know how their pictures are protected.