Feature Request: Using color for motion detection

Description

I have a problem with wind and trees causing motion and so I'm thinking about ways to adjust sensitivity using areas and colors.

For example, the area with moving trees is always some shade of green. Any amount of red in that area is something I would want to trigger on. But large changes in green shade or intensity should not trigger.

Perhaps the algorithm could learn what range of colors is common for each pixel and not trigger if values are in that range.

-- JonZeeff - 15 Jun 2008

Follow up

Comments

This was requested some time ago: http://www.lavrsen.dk/twiki/bin/view/Motion/FeatureRequest2006x05x24x075114

-- RomanGaufman - 15 Jun 2008

---

Here would be a practical way to try it. Instead of using "Y" (gray level), use (U/V) * 256. This should look for red/blue color changes instead of intensity changes.

Looking at the code, I can't quite figure out how to do this. But it should be simple.

-- JonZeeff - 15 Jun 2008

---

I should point out that with the current algorithm, a person wearing a red shirt and blue pants walking past some green trees might very well not be detected - because all the colors may be the same gray level and wind in the trees causes "motion" to be less sensitive in that area.

-- JonZeeff - 15 Jun 2008

---

I've been talking with Joerg Weber in IRC about implement some new algorithm using HSV / HSL :

http://en.wikipedia.org/wiki/HSL_and_HSV

But right now it's no a high priority and many others things requieres our effort , of course any help is very welcome

-- AngelCarpintero - 15 Jun 2008

I'm not sure that something that involves human color perception would help. The facts that I find most important are:

1) things that move in the wind are often green 2) clouds moving over the sun and wind moving shadows means that changes in intensity are prone to noise.

So I conclude "deemphasize green and intensity changes".

-- JonZeeff - 16 Jun 2008

---

All new algorithm for vision computer will be added as a plugin as it's scheduled in our roadmap, so you can use them in a pipeline.

-- AngelCarpintero - 16 Jun 2008

---

I think I'm starting to understand the code. YUV data is stored in 3 arrays, one right after another. The U and V arrays are 1/2 size since they are sampled less often. All are 1 byte per pixel. Motion currently uses the Y array for all analysis. Ie, it does nothing with color changes.

So all the functions that analyze the image need to 1) move further into the image buffer to access U and V and 2) account for there being 1/2 as many pixels, i.e. height * (width/2).

-- JonZeeff - 16 Jun 2008

---

Ok, I coded this up (very inefficiently) and it seems to work very well. Very insensitive to green, so my tree movement problems are gone. I used (U-V) instead of Y for all analysis.

U/V didn't work so well - when V is small, it gets very noisy.

-- JonZeeff - 17 Jun 2008

---

Could you post your code please? - sounds very interesting

-- RomanGaufman - 17 Jun 2008

---

What I did was really a hack. Before the motion detection code was called, I filled the Y array with the values of "U-V+128".

I've learned that there is a color space called "L*A*B" that takes into account human color perception. If the image was converted to L*A*B, the one would just look at the distance between the A and B values (called Delta E). Ie, sqrt( delta_A**2 + delta_B**2). This would measure "color change".

This wouldn't take into account that green movement is far more likely to be uninteresting motion (trees, etc). But it would help with changes in intensity (ie, currently sun goes behind cloud and motion is detected).

-- JonZeeff - 28 Sep 2008

---

Another enhancement I can think of would be to have an independent noise range for every pixel. Ie, if a pixel is within its normal range, then it doesn't add to the total of changes. For example, a pixel on the edge of a leaf will be very noisy as it constantly changes from bright white (sky) to dark green (leaf). But a pixel in the middle of a solid object is probably very stable and a change there is highly likely to be motion. Perhaps this is already done?

-- JonZeeff - 28 Sep 2008

---

Ok, I wrote a program to take several images and find pixels that change too much. You can take several images from your camera under no motion conditions and it will output a mask image file that excludes areas that have noise (ie, unimportant movement). Ie, this does what I mention one comment above. It does not do anything with colors or color changes.

What is the best way to submit this program?

-- JonZeeff - 29 Sep 2008

---

Jon, you can open a patch topic here to submit a patch. BTW: there is a feature already called 'smartmask' that masks pixels with high noise level.

The next step towards detection algo improvement will indeed be a change of the internal picture format from yuv420 to something else that makes it easier to process color information - most likely HSV, but I haven't started working on this yet. As soon as this is done, we can easily add shadow detection. That way we should get rid of the sun/cloud problem.

-- JoergWeber - 29 Sep 2008

---

Ok, I uploaded makemask. This is useful for masking off more occasional changes - a sudden gust on a calm day, cars driving by, etc. I don't believe that smartmask handles those. On the other hand, it is more precise than a manually, hand drawn mask.

-- JonZeeff - 30 Sep 2008

---