How Our Garden Webcam Spots Rain, Growth and Wildlife
A plain-English look at the computer vision behind Virtual Micro Gardening: image hashes, motion detection and pixel colours, and how a computer decides it's raining.
Every photo our garden webcam takes goes through a small piece of computer vision software, written in Python with the OpenCV and scikit-image libraries. It decides whether anything interesting happened, and if so, what. The results feed the game's suggested jobs, the wildlife collectibles and the garden diary. Here's how it works, with no maths degree required.
Step 1: Has anything changed at all?
Most photos of a garden look almost identical to the one before. Before doing any heavy analysis, the site calculates three image hashes for each photo. A hash shrinks the whole image down to a 64-bit fingerprint:
- An average hash records whether each part of a tiny, greyscale version of the image is lighter or darker than average.
- A difference hash records whether each part is brighter than its neighbour, which captures edges and shapes.
- A perceptual hash captures the overall pattern of light and shade, and it copes well with small changes like compression.
Two photos are compared by counting how many of the 64 bits differ, called the Hamming distance. A handful of differences means "basically the same photo", and the near-duplicate is tidied away. A bigger distance means something changed, and the photo is queued for proper analysis.
Step 2: What moved?
The analyser subtracts one photo from the next, pixel by pixel. Pixels that changed a lot become white and everything else black. Tiny specks caused by leaves shimmering or camera noise are ignored. So is a strip at the top of the frame, where moving clouds would cause false alarms. What's left are blobs of movement, and their size is the clue:
- A small blob is probably a bird or small animal.
- A large blob is probably a person or a large animal. Those photos are never shown on our public pages.
- A huge change is something major, such as the camera being moved.
Step 3: Is it greener or browner?
The analyser counts what share of the photo is green and what share is brown. If green goes up by more than a few percentage points, it's logged as growth. If it drops, the grass may have been cut. A rise in brown suggests bare soil or fallen leaves. It also measures how similar the two photos' structure is, a score called SSIM, to catch things being moved around.
Step 4: What's the weather doing?
The top half of the frame is treated as the "weather band".
- Rain shows up as lots of tall, thin streaks of change. The analyser only calls heavy rain when there are enough streaks, they're long and thin enough, they cover enough of the frame, and the sky is cloudy.
- Lightning is a sudden, bright flash across a large part of the sky while cloud cover is rising.
- Overcast is when cloud cover jumps up while little else in the garden moves.
Why the thresholds matter
Every one of those rules has a number attached: how big a blob, how many streaks, how much greener. Set them too low and every gust of wind is a "visitor". Set them too high and a real hedgehog walks past unnoticed. Early versions of our rain detector fired on almost any hazy frame, which kept telling players off for watering on dry days. Tuning these numbers against real photos is an ongoing job.
Try it yourself
If you're learning to code, this is a great first computer vision project. Take two photos from the same spot and subtract them with Python and OpenCV to see what changed. Then try writing your own rule: how would you detect snow, or autumn leaves?
Then see what our camera has spotted lately in the garden diary.
Ready to look after a real garden?
There's no subscription and nothing to pay. It runs in your browser and only takes a few minutes a day. Ages 13+.