Everything Got a Name, and the Planet Became a Database

THINGS THAT MOVE

Grids and curves work beautifully for things that stay where they are. A field does not wander off between satellite passes. A stork does. So does a whale shark, a container ship and an aircraft at 11,000 metres.

For anything that moves, the position is the interesting part, the identity has to survive from one sighting to the next, and the data only makes sense as a sequence.

Volume 2 described the tags, cameras and microphones that watch animals. What they produce is a scatter of positions, photographs and sound files, which is not yet a record of any animal's life. Turning that scatter into data took agreements about how to store a journey and how to recognise the same individual twice. Machines, meanwhile, arrived at the same answer from the opposite direction, by announcing their own names.

TRACKS

A journey became a table

A tag on a stork sends back a position every few minutes or hours. Each fix alone is two coordinates and a time. Kept in order under the same bird's name, they become a track, and a track is a very different thing from a map of where a species lives. It is a biography: this bird, this route, the year it left three weeks late.

Tracks from thousands of studies needed a common home, since a stork followed in Germany and one followed in Spain can only be compared if both are stored the same way. Movebank, a database run by the Max Planck Institute of Animal Behavior, became that home, with a shared format for animal, tag, time and position. It now holds on the order of 6 billion locations from more than 1,400 species.

The shared format opened up questions nobody could ask of a single study. Do young birds follow older ones? How did a route shift over a decade of warming? What did a whole population do when a new road or wind farm appeared across its path? A range map drawn as a coloured blob averages all of that away. Tracks keep every individual's story intact, and the blob can always be drawn from them afterwards.

IDENTITY

Recognising a whale shark with software built for stars

Tags are expensive and only work on animals someone can catch. Many animals carry their own identification instead. The pale spots behind a whale shark's gills are unique to each individual and stay the same for life. The Australian marine biologist Brad Norman spent years photographing the whale sharks of Ningaloo Reef and matching their spots by eye, which worked in the way that reading every page of an archive works.

The breakthrough came from astronomy. Norman teamed up with the software developer Jason Holmberg and the astrophysicist Zaven Arzoumanian, who noticed that a whale shark's spots look a great deal like a field of stars. They adapted an algorithm written in 1986 by Edward Groth to match star patterns for the Hubble Space Telescope, adding corrections for the fact that a shark, unlike the night sky, is curved, moves about and is usually photographed through murky water. The result became Sharkbook, where divers anywhere can upload a photo and receive a ranked list of likely matches for a person to confirm. Twenty holiday snaps taken by twenty tourists over ten years can now add up to the life history of one shark.

The idea spread to zebra stripes, giraffe patches, the ragged trailing edges of whale flukes and the rosettes on jaguars. In every case an animal becomes a record the moment it has a permanent identifier and a way of comparing each new sighting with all the old ones.

Camera traps and microphones raised the same problem at a coarser level. A camera trap does not know what it photographed. It knows something warm moved. A recorder does not know what called. It knows the air or water shook in a pattern. The animal is locked inside the photo or waveform until someone attaches a label, and labels needed rules of their own: which species list, which name when taxonomists disagree, what to write for a blurred tail that might be a jaguar, and whether two photos a minute apart are one visit or two. Darwin Core, built for specimen drawers, fitted here too, so a camera-trap detection in Brazil can sit in the same table as a pressed plant at Kew. The pile of unlabelled photos and recordings grew far faster than anyone could label it, and reading them at scale would take an entirely different kind of machinery, which belongs to Volume 4.

SELF-REPORTING

Machines that announce themselves, and everything else that reports

Animals have to be tagged or recognised. Aircraft and ships simply say who they are. Under ADS-B, Automatic Dependent Surveillance Broadcast, an aircraft works out its own position by satellite navigation and broadcasts its identity, position, altitude and speed several times a second, to nobody in particular, where previously the ground had to find it with radar. AIS, the Automatic Identification System, does the same for ships, sending identity, position, course and speed so vessels can see one another and coastguards can see the traffic.

Both were required for one reason: to stop collisions. Both turned out to be planetary datasets. AIS now underpins fisheries monitoring, because a vessel's movements reveal whether it is travelling or fishing, and a ship that switches its transponder off inside a protected area has said something by going quiet. Aircraft data has been used to study contrails and to watch aviation collapse and recover during the pandemic. Rules written to stop things bumping into each other produced, as a side effect, a continuous record of where the world's traffic goes.

These records arrive with their names already attached. A satellite pixel has to be told what it contains, and a whale shark has to be recognised from its spots, but every ADS-B message carries the aircraft's permanent 24-bit address and every AIS message the vessel's nine-digit maritime identity number. A ship's track across an ocean is not a scatter of sightings someone has to join up. It is one object with a name, a position every few seconds or minutes and a history going back years, among the cleanest records on Earth.

A modern combine harvester records the yield continuously as it drives, drawing a map of the field at a detail no satellite approaches and pinned to position by the same satellite navigation that made coordinates free. Tractors log what they spread and where. Individual farms now produce ground truth of a quality agricultural remote sensing spent decades trying to approximate, and it mostly sits in private hands under commercial terms.

Electricity grids measure themselves to keep the lights on, and their demand curves turn out to be one of the sharpest records of human activity, heat stress and economic shocks. Air quality went mobile as well. Aclima's sensors, mounted on Google Street View cars, recorded pollution roughly once a second while driving and showed concentrations changing sharply within a few hundred metres, differences a single fixed monitor in the city centre was never positioned to see.

Phones are the most complete layer of all, and the most delicate. Aggregated movement data has been used to study evacuations during disasters and the spread of disease, and it is the one dataset in this book where the thing being measured is a person who never chose to be measured for that purpose.

This layer has a shape of its own. It is thick wherever there are aircraft, shipping lanes, well-capitalised farms and city budgets, and thin elsewhere, because it records human machinery rather than the planet. What it added was new all the same: a continuous description of human activity with names, places and times built in. By now a stork, a shark, a ship, a field and a pixel could each be described precisely. Getting them into one place, where they could be asked about together, was the last step.

SHAPES AND IDENTIFIERS

Four shapes, and the same thing recognised every time

Put everything so far side by side and a structure appears that nobody designed. Grids hold things that vary smoothly across a surface: temperature, reflectance, height. Shapes hold things with edges: a field, a river, a national park.

Networks hold connections: roads, power lines, a migration corridor. Streams of events hold things that happen at a moment: a lightning strike, a ship's position, a deforestation alert.

Almost every dataset about the planet is one of those four, or a join between them. That is what it really means to say Earth became computable. It does not mean anyone understands it. It means people agreed on a small number of shapes and persuaded the world to fit into them, with the metre, the lawn in the box, the Greenwich meridian and the whale shark's spots all doing their part.

For any of this to add up, the same thing has to be recognisable across observations. A field needs a permanent identifier so this year's harvest attaches to last year's soil test. A ship needs one so its track is a voyage rather than a spray of dots. A whale shark needs one so twenty sightings become one life instead of twenty sharks.

An identifier turns a measurement into something that can be updated. Once a thing has a name, a position, a current condition and a history, it stops being an observation and becomes an entity a system can reason about, which is the starting point for every forecast and every learned model that follows.

It is also the least standardised part of the whole arrangement. Ships and aircraft were given their numbers by treaty. Forests, rivers and fields were not. There is no universal way to say that this is the same forest as last time, and a remarkable share of practical work with Earth data consists of matching up identifiers between systems that each invented their own.

COMPUTING NEXT TO THE DATA

The archive outgrew the download button, and a record is not a forecast

For most of this history, using data meant getting a copy. That stopped working once archives grew so large that moving them took longer than analysing them. The answer was to turn the arrangement round: leave the data where it is and send the calculation to it.

The turn became public in December 2010, when Google launched Earth Engine at the UN climate conference in Cancún, parking the Landsat archive next to an enormous number of computers and letting researchers analyse decades of imagery without downloading a single scene. Space agencies and other companies built similar platforms. A question that once meant years of ordering, shipping and storing imagery, such as how much forest every country lost in each year of this century, could now be asked in an afternoon, and in 2013 a global map answering exactly that was published using Earth Engine. The archive became something you query rather than something you own.

Old paper records got the same treatment. KoBold Metals, a mining exploration company backed by investors including Bill Gates and Jeff Bezos, built its business on gathering a century of scattered geological material, old drilling records, chemical surveys, field reports and satellite images, into one searchable place and running machine learning across it. In Zambia the approach pointed to the Mingomba copper deposit, now a $2.3 billion development that the company says went from discovery to development in about five years, with an expected output of at least 300,000 tonnes of copper a year. The measurements had existed for decades. What was missing was a way to hold all of them in view at once.

The arrangement has a consequence worth saying out loud. Whoever hosts the archive and supplies the computing decides which questions are cheap to ask. The free data policies Landsat and Copernicus fought for, told in Volume 2, opened up the measurements. Opening up the means of processing them at planetary scale is the next round of the same argument.

In one sense the job is finished. A planet whose rivers, soils, animals, harvests, ships and cities all carry names, coordinates, conditions and histories is a different object from the one Aristotle was reasoning about, and the difference is not one of degree. The farmer's valley knowledge that died with the farmer now has somewhere to live.

In another sense it has barely started. A database of everything that has happened contains no opinion about what happens next. Every choice made along the way is still inside it: the averaged pixel, the lawn in the box, the coordinate with more decimal places than the observer deserved, the places where nobody could afford an instrument. The next step feeds all of it into equations that assume the numbers mean what they say.

Earth became readable by machines first. Whether it could also be calculated, whether the numbers could be pushed forward to say what the planet would do next, had already been bet on in 1904 by a Norwegian physicist who decided the weather was simply a problem of starting conditions.