Motion - Statistics Data Patch
You are here: Foswiki>Motion Web>StatisticsDataPatch (22 Jul 2008, SimonW)Edit Attach

Getting data related to the motion detection over time

Description of Patch

This patch is a follow up of the FeatureRequest2005x10x21x025159

It will provide some statistics reporting for individual Motion threads to the Motion HTTP Control page.

Installation of Patch

Download the patch file. Copy it to the motion source directory and issue the following command.

patch < motion-20051024-051001.diff

Then re-build Motion and test the patch.

Change History of Patch

  • 1.0 Initial revision - This patch will provide some statistics reporting to the Motion HTTP Control page.
  • 1.1 patch - Altered how the total Different pixels data gets generated, removed superfluous total_events variable, added a "back" on the HTTP Control page.
  • 1.2 patch - Rearranged, subtlely, the location of the per-thread data, and added counting of Frames-Seen-So-Far. From this one can calculate the framerate of the thread.
  • various versions up to...
  • 20070306 Statistics web page and throttling ability for netcams if frame_limit is less than the arrival rate of frames from the cam. The amount of netcam throttling is reported on the Statistics page. Other useful items are reported, including MotionThruPut which is the number of frames actually processed for motion-detection each second. (This is great for working out if your machine is fast enough for the task).

Discussion and Comments


Here are a few thoughts on my implementation, right off the top of my head. In the HTML output mode I can very easily TABLE-ize the data so that it fits nicely within a browser. That's a possible improvement. I'm not sure about my use of "unsigned long"'s for the data types. Someone more experienced may know better whether this is a good idea. I feel that an unsigned, very large number, is pretty much required for, at least, the "Total" value (total_diffs). This is probably not as necessary for the "Events" value (total_events).

-- PeterS - 25 Oct 2005


What is the difference between cnt->total_events that you introduce and cnt->event_nr which is already there?

The place you increate the counter is not very logical in the program flow and since we already have cnt->event_nr which is increated in motion.c 1079 you already have this data.

Then there is this

cnt->total_diffs+=olddiffs;

olddiffs is 0 unless despeckle feature is used so you cannot use olddiffs - the stat feature should not be depending on the use of another feature unless it is the despeckle feature you want to measure (in setup mode you actually get this info).

And why would you want to count all those single noise pixels that the despeckle feature later removes?

If you count diffs it makes more sense. But even just adding up all the changed pixels makes no sense to me. What will you use this information for? It cannot be used for tuning. If you want statistics for how much activity you have in an area you do not want to count noise pixels in the dawn and dusk. And you do not want the counts from some leaves on a tree which the smart_mask has masked out.

Activity should be equivalent to Motion detection. And then you can add the diffs value (there is a conversion specifier for it) to the data you put in MySQL and make great statistics based on that. Then you can also restart Motion without loosing your statistics.

If we do a statistics feature in the httpd control I would think more relevant data could be shown. Events per hour. Motion frames per hour. Diffs per hour. Maybe previous full hour and running in current hour.

-- KennethLavrsen - 26 Oct 2005

These are good points. I've altered the patch so far to take into account some of this. cnt->event_nr is currently a programmatic counter, simply to show which event we are in and also if we are in one, at least it looks like. To use it as a "The number of events that have occurred since Motion started" I have to subtract 1 from it, but this is not a big deal.

I've altered the using olddiffs. I originally was using just plain diffs but found that, for some reason, the cam I was testing with would not register any diffs at all unless there was motion associated with it. I'm not entirely sure this is the case anymore, however, as my total_diffs does actually increase even though my event_nr does not, so this is satisfactory.

I plan on using the diffs counts to graph pixels changed over time. Although a Motion cam's (thread's) threshold is a per_frame value and not a "pixels changed per second" value it is difficult to use these numbers for tuning but that is outside the scope of this patch, really. By counting the # of pixels that have changed since the last time we looked at it we can still get some useful data.

You are right, I think I'll add more Statistics/Time data to this as it can be relevant as well. Like you said, "Events per hour, Motion detected frames per hour" etc.

Thank you for the comments, I'll be working on this. I'll also likely be adding a "Motion started on (date) and has been up for (time)." line to this page. That would be a statistic and would be easily calculated.

This patch reflects some of this.

-- PeterS - 26 Oct 2005

I've setup a quick test of the statistics logging. Please refer to the following page.

http://petersmith.info/motion-stats.html

-- PeterS - 26 Oct 2005

So far I'm only using data that has already been provided. By collating this surfaced information someone can calculate, on the backend, some extra data. By doing this I'm keeping the computational load on Motion to a minimum. My patch currently only adds data and moves it around a bit.

-- PeterS - 27 Oct 2005

Nice statistics demo.

-- KennethLavrsen - 29 Oct 2005

Peter nice feature wink

I've updated the patch against last daily snaphost and fixed a warning replacing %ul bu %lu .

btw , i guess that use cnt->diff in statistics is useless for general use because it'll have a value only if the stats are requested while a motion is detected. It only has sense for debug purpose ... am i wrong ? Maybe it might be better add a new variable to store the lastest diff when motion is detected.

-- AngelCarpintero - 30 Nov 2005

Angel: That is the beauty of it. Every time motion is detected, it adds the amount of motion to an always-running-total. That way, you can ask it for a value at a given time, which we'll call X. Then, after T1 time has elapsed, you can ask for the value again, which we'll call Y. Now, you know how much motion has occurred through the period T1 by looking at the results of Y-X.. So you can look at it every 5 minutes, 1 minute, 30 minutes, etc. This is the basis of MRTG, for instance. It would make the most sense if (and when) I produce some php and/or perl scripts that could come packaged with Motion that would handle this datalogging and provide the appropriate graphs/data to whomever actually wants it.

Another thing I'm going to have to add would be timing statistics on the different portions of Motion running. And also statistics on memory/frame usage. All too often users are asking "why is Motion dropping frames?" when you can only answer in generalities. If we had the statistics that showed that the creation of movies is taking too long, or perhaps the access of the netcam is too slow and/or erroring, or maybe they are using too large a precapture value--basically provide the data so that users can support themselves. We currently do not have these numbers. Maybe Motion needs more CPU? Maybe it needs faster disks? Maybe it needs a gigabit uplink? More RAM perhaps? I'd like statistics to answer these questions!

-- PeterS - 14 Dec 2005

Angel: Btw, thank you for fixing that bit in my patch--I often get "ul" and "lu" mixed up!

-- PeterS - 14 Dec 2005

Need to add CPU usage per thread. Also, need to identify which thread is which, that could help out immensely.

-- PeterS - 30 Dec 2005

Also need to add the current cam status, such as "CONNECTION TO CAMERA LOST SINCE

-- PeterS - 12 Jan 2006

I was actually thinking about creating an event when connection to the camera is lost. But I did not want to implement it until all camera types were supported. Not only Netcams

-- KennethLavrsen - 12 Jan 2006

I see benefit to having both.

-- PeterS - 13 Jan 2006

In the latest patch I play around with trying to get at the pid of a thread if it exists (doesn't seem to on FC4, but I have seen pids for children on RHAS...) Also I'm lining up to try and get at the per-thread processor usage, but that's going to be weird too. Look at this on my FC4 box:

Results of a ps auxmwwww

root      9843  176  7.4 138704 77532 pts/6    -    23:08   3:01 /usr/local/bin/test4/bin/motion
 root         -  0.0    -      -     - -        Sl+  23:08   0:00 -
root         - 81.5    -      -     - -        Rl+  23:08   1:24 -
root         - 90.5    -      -     - -        Rl+  23:08   1:33 -
root         -  0.0    -      -     - -        Sl+  23:08   0:00 -
root         -  2.2    -      -     - -        Sl+  23:08   0:02 -
root         -  2.1    -      -     - -        Sl+  23:08   0:02 -

The above shows what cpu usage looks like with simply two netcams. I would like to be able to surface these statistics as well.. Working on it..

-- PeterS - 13 Jan 2006


I have worked on extending this patch a little, and here are 2 patch files.

The first is a very simple patch, to webhttpd.c only (SWpatch_Simple-fps), which reports the fps (calculation method explained below) in each thread's Detection Status page. This patch is purely to test the calculation system, which is the following:

- if a V4L device, the fps is given by the variable 'lastrate'

- if a Netcam, the fps is calculated by 1E6/'calculated frame time'

It is not necessary to apply PeterS 's patch before this one (this patch does not provide the Statistics page, just a mod to the Detection Status page). I cannot test with Video4Linux (I only have netcams), so if someone can report back if the 'lastrate' figure is good to indicate fps, that would be welcome.

The second patch (SWpatch_Statistics20070129) takes the same algorithm and extends the 'Statistics' page (not normally present in Motion, is added by PeterS 's patches) to include the following items:

- time of Statistics page generation in server

- camera thread's text_event string which I set to fixed text like "Front Porch"

- fps (calculated as per above)

- lastrate (presented as well, since my netcam sometimes gives 1,2, or 3 frames in 1 second)

- Connection Lost status and if so, the Connection Lost time as found on the grey screen.

The same information is presented in http and raw text output modes (although raw text output mode has not had any testing). I plan on adding 'Motion has been running since...' information as well. The second patch does not need PeterS 's patches to be applied first, as it's all included.

This is all food for others to modify, as each will probably have their own requirements for appearance and statistics.

My next change is to make the page self-refreshing at some time interval using multipart/replace technique. I find pressing 'Refresh' tiresome to keep an eye on the fps of my netcams.

-- SimonW - 29 Jan 2007

Another revision of the Statistics patch, SWpatch_Statistics20070130, with a couple of important changes....

I found that the original calculation method, using calculated_frame_time was totally wrong. Calculated_frame_time is a variable used to adjust the delay time in the timing loop, and as such it always approaches 1.0.

A new method of obtaining fps is introduced, where the total_frames is saved at 1 second intervals, and the 'new frames per second' averaged over a set period, currently 10 seconds (but could be changed). It was quite a bit more coding but is accurate, and if required, all the per-second numbers could be printed to screen, or things like min/max/stdev calculated.

This average fps over 10 second period is logged to the screen and logfile in Setup mode, and is presented on the web server Statistics page. It starts at 0fps and after 10 secs is fully valid.

Another new item in this patch is - in Setup mode, if the delay time required is zero, ie. there is no more 'spare time' available, a message is reported that the frame rate is too high. If there is no 'spare' time, then the desired frame rate will not be maintained as other CPU tasks etc, are scheduled.

The Simple-fps patch has been removed since it used the old method. It cannot be used with the new method, as there were quite a few changes in other source files, ie. it's not so simple now.

I still have some tweaking of the data presentation, etc to do, but I think the figures are right. It will be helpful in characterising my webcams and the keep-alive patch.

-- SimonW - 30 Jan 2007

This is the best revision so far. It is a patch for svn166.

It now shows the framerate each second, for the last 10 seconds, for each thread. The fps figure is now correct (found some silly errors in previous versions). 'ReqRate' is the framerate specified for the thread in the .conf file, so you can compare what's configured with what you're actually getting. All 3 lines include a link for the control page of the thread (eg. 'Thread1' appears 3 times). This is so I can still 'grep' the page to find the Events= figure in my backend processing.

And as described above, the apparent 'camera name' (eg. FRONT PORCH) is simply printing the config option 'text_event'. In this, I do not use any conversion specifiers (eg. %f). If I did, they would not be translated into real data, but left as '%x'.

Patch SWpatch_Statistics20070201 does not need PeterS's patch applied beforehand and it looks like this:

Pointing browser to :8000/statistics
Statistics_20070201.jpg

One bug still remaining, the 'Generated at ' time is broken. I think the code will need checking through to see how it would cope with conditional compiles --without-V4L etc. I haven't got to try making the page a multipart (self-refreshing type) yet.

-- SimonW - 02 Feb 2007

Revised again. Now, a non-functional netcam does not have an increasing 'Frames' count and a looking-real fps number. Previously, they were indicators of 'attempts to get frames' and not 'successful frame reads'.

The 'Generated' time display has been fixed.

A further improvement would be to move the Statistics page into a subroutine as currently it's generated in a block in an If statement.

-- SimonW - 05 Feb 2007

When making the last revision, I broke the Frames reading. I have realised that a V4L device and a Netcam must use different calculation methods, and am testing a new revision to handle this. I will post it when it's ready.

-- SimonW - 08 Feb 2007
stats1.jpg

Good work Simon wink

But seems that something is not working in the first field, just before (Video4Linux).

Btw : Would be nice to set V4L or V4L2 instead of Video4Linux.

-- AngelCarpintero - 08 Feb 2007

Thanks Angel.

The % % %.... is your 'text_event' config option being still set to default (in the config file). Conversion specifiers are not supported. I have set my text_event fields to a friendly camera name, eg. 'Front Porch'.

If you think a separate config option holding a text specifically for camera name, then it could be added.

Yes, I hear you on the V4L/V4L2 output. However I have no v4L cameras, but I have added some code in webhttpd.c to describe V4L1 or V4L2. I have left it commented out - it won't compile as I don't know how to access the viddevs struct from webhttpd.c. We need that, or an alternative method, to find the V4L type. If you can help, it would be appreciated.

This is the current look: Statistics_20070209.jpg

I have solved the problem I mentioned with the fps reading for netcams, I used completely separate calculation methods for V4L and netcam, since some of the variables don't report quite the same thing in the netcam case. The fps reporting seems much better now. The Req.Rate number takes account of minimum_frame_time, if set, so can be less than 2.

The patch is posted below (SWpatch_Statistics20070209) - however, please take care with it. It has been hand-edited to remove additional diffs, so the fuzz factor required may be high. The patch is now against SVN166.

-- SimonW - 09 Feb 2007

Ok , i'll introduce the V4L/V4L2 feature and make a patch against last svn version.

Thanks for your effort smile

-- AngelCarpintero - 09 Feb 2007

Thanks smile It's been fun, helping to make Motion better. The stats facility really helps me get a handle on a slow server running 2 netcams, neither of which are very fast either.

-- SimonW - 09 Feb 2007

I've added V4L/V4L2 feature and upload a diff against latest svn version ( r171).

-- AngelCarpintero - 10 Feb 2007

Great. I am now using that patch in my 24/7 server and will keep an eye on it for some days.

I am now looking into the fact that my netcam is captured at about 5fps, but Motion does not detect 'fast' events. It still misses some. Perhaps, again, this is down to the low CPU horsepower of my server.

If I can find a metric of how well motion 'could be detected' for example, motion-fps (not camera-fps since they seem to be in different processes) then I'd like to add that.... it helps us answer the question: "Why is my motion detection unreliable?"

-- SimonW - 11 Feb 2007

The patch seems to work quite well with V4L/V4L2 devices all values are quite exact.

The answer to your question is not easy there are several situations that could affect, there're some parameters in motion to try to filter out some false detection and others that have to be tested & tuned ( i.ex : despeckle, noise_level, threshold and contrast , hue , etc ... for v4l/v4l2 devices ). Setup mode is a nice help but will be great if we can extract from stats what are the most related parameters.

-- AngelCarpintero - 12 Feb 2007

Right, I have been doing some investigation into the jpeg sub-section. The jpeg decoding runs as a thread, taking a new frame from the netcam (or V4L) when it needs one. On my slow server I only get approx 1 frame decoded per second, even though the netcam is supplying frames at about 5 fps.

I added some more statistics to the patch to measure the rate the actual Motion comparison is made in a 6 second interval. This gives a number which is the 'Motion fps', whereas the netcam thread is getting frames in at the 'Camera fps'. The Motion fps is shown as MotionThruPut on the statistics page.

See the 2 examples, firstly on my slow server (bogomips=230): slowserver.jpg

And on my Sempron 2400 (bogomips=3300): fastserver.jpg

The Sempron machine (which is over 10 times faster) has no problem maintaining a MotionThruPut of 4.0, that means 4 frames per second are compared looking for motion.

It seems my earlier suspicion about my slow server was correct, it is processor-bound but not really for the netcam, for the frame decoding / despeckle / motion compare also. This explains why, when motion is detected by my slow machine, the resultant mpeg file has frames about 1 second intervals, and irregular, while the motion is occurring. I guess the 1 fps is not always exact.

The latest version of the patch is attached. I tidied up a little also. Sorry I missed out rev (6)! Still to do is the non-html output mode, needs to be brought into line with the html output 'look'.

-- SimonW - 13 Feb 2007

I am coming to the end of my work on the Statistics patch now. A few more fields have been added, explanations below.

There is also code included to throttle netcams. I found that if I only wanted 0.5fps (ie. minimum_frame_time = 2) in one thread, then the motion processing took up time which other threads could make use of. On my slow server this is important as I run 2 cams and would ideally like to run 3 (don't know if this is achievable).

So now, there is code included to detect the desired fps (using frame_limit and minimum_frame time as appropriate) and if the cam is delivering frames faster than needed, to dump frames after they are read in - but before they are jpeg-decoded. This saves CPU time, especially useful as my slow server is an ARM CPU which uses softfloat (ugh).

The throttling process works on a 10 second time frame, beginning 10s after start-up (if needed). Debug output is produced in Setup mode to observe its operation. It is not 100% accurate and will sometimes produce an fps +/- a few fps either way. My netcam can be quite variable in its output rate, which the throttling loop does not track totally. Throttling cannot be turned off in this version, but that could be added. Simplest way to disable it is to set frame_limit greater than the cam can produce.

Here are screen shots of it operating:

A 320x240 image on the netcam produces 12fps which is throttled to achieve 3 fps: throttle_320x240.jpg

A 640x480 image on the netcam arrives at only 3fps which is throttled to achieve 1.6fps over the last 10 seconds. Note that as netcam output rate varies the throttle system takes 10 secs to respond - this leads to slow movement in the average fps number. throttle_640x480.jpg

I have also added fields for Gap (the configured gap time), InEvent, which is simply whether an event is being handled right now, and GapTimer, which increases during an event until it reaches the Gap value, then returns to zero. It is helpful when observing motion working to see these numbers. I also added a Size field so you can check your netcam is set to produce the desired frame size (and for me, when I reduce it, the processing rate increases).

Throttling should enable larger numbers of netcams to be used, since the Jpeg-decompress and Motion processing is not done continuously. Throttlng allows the Jpeg decompress to be done once a second, for example, even if 10+ frames arrive from the netcam per second, when on a fast machine it would otherwise do 4 (for example) per second. This should be seen as a lower 'load average' on systems with a number of netcams.

Some could say there should be separate patches, but it helps understanding what Throttling is doing when the Statistics are also there. It should be possible to make separate patches when it's tested fully.

I will have to also test the throttling works on the fast (Sempron) system also.

-- SimonW - 21 Feb 2007

Now I have some throttling comparison using the Sempron 2400 system. I compiled Motion svn 172 with my Stats and Throttling patch, and ffmpeg snapshot 2006-11-30 which I use on my other server too.

The first picture is running 4 netcams, as fast as possible (all frame rates set to 50). Load 1.08 and CPU for Motion was 50% (using 'top') snapshot_load1.08.jpg

The second picture is just the same setup, but with fps set to 2.0 on all threads, and the throttling was operating. Load 0.58 and CPU for Motion was 15% using 'top': snapshot_load0.58.jpg

So it does show that CPU load can be much reduced by throttling a netcam. I have made a couple more changes since the last posted patch, and before I post another, I want to prove that motion is still detected ok.

-- SimonW - 28 Feb 2007

I now know that motion is detected ok when throttling a netcam. So it's good to use.

The change(s) I made were to re-calculate the throttle parameters once a second rather than once every 10s. It is possible to throttle so much that no frame is available when the jpeg-decompress has finished; if this is the case a "netcam_init_jpeg returns 6" will be logged. This is harmless and could be made 'quiet' easily.

The updated patch is posted below, SWpatch_StatsThrottle20070306.

For new users coming to this patch, if your 'text_event' config item is still set to default, you will see %D%f for example on your Statistics page. I have set mine to 'friendly names' for the cameras since I don't use text_event for any other purpose. Sadly, conversion specifiers (%H etc) are not supported and just appear as %H etc.

I will try to get one further patch posted sometime to do the following: * Silence the occasional 'netcam_init_jpeg returns 6' debug message * Tidy up the 10 seconds worth of fps data. Perhaps 6s is enough, and it would not wrap onto a new line often. * Make the text statistics page the same as the HTML one * Add a link on the statistics page to view the current webcam port (if available)

I probably won't get the time to separate the throttling from the statistics patch - but that could well be of limited use. It is useful to know if/when your netcam is being throttled, and that can only be observed on the Statistics webpage at present.

Based on my discoveries about my slow server (Linksys NSLU2 with Openslug), I am moving on to building a faster Motion server using a EPIA mini-ITX motherboard (bogomips ~2000 for 10 Watts of electricity). Should be fun! When there is some news, I can write a Wiki page about non-PC platforms.

-- SimonW - 06 Mar 2007

My almost-last patch for Statistics for now, is attached ... "SWpatch_StatsThrottle20070323".

It covers the changes mentioned in my last post:
  • Silence the occasional 'netcam_init_jpeg returns 6' debug message. Now silently ignored.
  • Tidy up the 10 seconds worth of fps data. Six seconds are presented, so it does not wrap onto a new line often. Calculation time is now also 6 seconds not 10.
  • Made the text statistics page the same as the HTML one
  • Added a link on the statistics page to view the current webcam port (if available). This last feature contains a derivation of the local IP address.

I plan on making one further feature: to make the throttling netcam code respect the setting of low_cpu. This will allow low frame rates when motion is not detected, then rising to a 'full' frame rate when it is. (The 'full' frame rate can still be throttled).

I will keep an eye on this page for reports of bugs and/or successful testing. Thanks!

-- SimonW - 23 Mar 2007


I have now added the one remaining feature I thought necessary: support for low_cpu on the throttling of a netcam.

Now, the throttling code will take into account the setting of low_cpu when calculating the desired fps. If motion is not being detected, the low_cpu figure is used for fps. (Disable this feature by setting low_cpu = 0). When motion is detected, within 1 second the netcam fps rate rises to the figure defined next.

If low_cpu equals 0, or it is set non-zero and movement actually is being detected, the frame rate is calculated as follows. If minimum_frame_time is non-zero, the frame rate is 1 / minimum_frame_time. Else the frame rate is equal to framerate (in source this is the structure element frame_limit).

Any of the fps rates above can be throttled-down from the 'raw fps' of the netcam. Also please note that throttling is only possible on a Streaming netcam (one using video.cgi or similar URL). Non-streaming netcams have their throttling handled by motion already, using the frame_delay calculation. Throttling is approximate and can only usually obtain an fps within 1fps for high figures, or about 0.2 fps for low fps. It also can take a few throttle-update cycles (6 secs each) to fully stabilise at a new fps figure.

I have tested this new feature briefly and it works for me. In my 'production' motion setup, I do not use low_cpu (I may do in the future), but am using the patch itself (I have low_cpu=0). I have left the previous patch version (20070323) posted, in case of problems.

The Statistics HTML page has been slightly modified, it now reports the values of motion_detected and low_cpu to the screen to help work out 'what's going on'. That forced me to use another screen line per thread.

The latest patch is posted below, it is SWpatch_StatsThrottle20070411. Against svn174 but should work with most recent svn versions. And a screen shot of it working, named throttle_lowcpu.jpg.

-- SimonW - 11 Apr 2007

An update on what I'm currently working on.

I am using this patch in my production server, together with OneFFMpegWithEventNumbersUpdatingPatch to monitor 3 cameras. It's running on Grafpup Linux and has been successfully running since May 2007.

I've written a collection of scripts to produce daily HTML reports of Events per camera, and to compress the MPEG-1 files Motion generates (I use these to get crash- and power-cut tolerance) into OGG Theora files daily, at night.

I am using an EPIA M10000 motherboard which is also a server for other things. It handles the load very well and no slow-downs have yet been noticed.

I plan to write a page on here soon to present my experiences of using three platforms for Motion: Linksys NSLU2, EPIA M10000 and Athlon 2400 level.

So, I am still very much around. I have also noticed that the StatisticsDataPatch (later versions only) have a fatal error if used on a machine with a V4L camera. I plan to debug this. But for now I am using the patch 24/7 on a netcam-only system.

Greetings to everyone.

-- SimonW - 08 Aug 2007


I have updated my patch to work against SVN223 (and probably later ones too) where low_cpu has been removed. No other functionality changes, just to make it compatible with the more recent SVN versions. It is SWPatch_StatsThrottle20070928_svn223 below.

I still intend to locate the bug which makes this not work on a V4L system. It is perfectly usable on a netcam system. Probably a preprocessor directive problem, I think. But I have to put a capture card back in a machine to do so. Soon!

-- SimonW - 09 Oct 2007

Posting the latest StatisticsDataPatch which has printout of the TCP receive and transmit buffers. I still haven't solved the issue mentioned above with V4L under this patch, and am posting it here mostly so I can download it onto a test system to try to fix that bug!

I also have another problem with this patch, after 2 days' running, the TCP buffer sizes no longer display since an error is raised in the kernel log: getsockopt for receive buffer returned error: Bad file descriptor. I suspect that the fd stored in the context structure gets out of date, since the TCP port used (seen with netstat command) seems to change after long runtimes. That may be it, or it may not.

-- SimonW - 19 Jan 2008

Just reported a slight issue with the Keep-Alive code in FeatureRequest2007x01x22x231542 and the point where getsockopt begins to fail is where the Keep-Alive ceases.

I expect the solution to that issue will help this one, so I'll report back. Simon.

-- SimonW - 22 Jan 2008

Yes, the solution to the issue with the Keep-Alive code has also solved the problem here.

It's also worth noting that when a netcam has Keep-Alive enabled, the Statistics Data Patch can display the buffer sizes, because the socket id is stored in the context struct in a persistent manner, and changes when the socket changes (as Keep-Alive periods end and are re-started). The problem noted on 22/01/08 occurred because the Keep-Alive code could not resume after the initial Keep-Alive period ended - it continued in non keepalive mode. Thus, it started closing the socket immediately after 1 image had been received from the cam (standard non-streaming and non-keepalive action), and the context struct contained an invalid socket id.

It's working now, with no changes necessary.

Simon.

-- SimonW - 08 Feb 2008

Just a note to say that I have no longer a machine with a V4L capture card, so will not be able to debug why this patch does not work on a V4L system. If someone else could look at it, that would be great. Just carefully reading the code, or looking at the pre-processed C code output, in a V4L enabled compile, might reveal the bug. As I said, I suspect it's a preprocessor directive problem - I've included a bit of code within an IF_V4L type directive which should not be.....

Simon.

-- SimonW - 22 Jul 2008
I Attachment Action Size Date Who Comment
SWimg_Statistics20070201.jpgjpg SWimg_Statistics20070201.jpg manage 76 K 02 Feb 2007 - 13:01 UnknownUser Image of the Stats page 20070201
SWimg_Statistics20070209.jpgjpg SWimg_Statistics20070209.jpg manage 44 K 09 Feb 2007 - 01:31 UnknownUser Image of the Stats page 20070209
SWpatch_Statistics20070210.diffdiff SWpatch_Statistics20070210.diff manage 15 K 10 Feb 2007 - 01:27 AngelCarpintero Statistics page revised(5)
SWpatch_Statistics20070213.diffdiff SWpatch_Statistics20070213.diff manage 16 K 13 Feb 2007 - 07:41 UnknownUser Statistics page revised(7)
SWpatch_StatsThrottle20070220EXT SWpatch_StatsThrottle20070220 manage 23 K 21 Feb 2007 - 23:29 UnknownUser Statistics page revised (8) with Throttling ability for netcams
SWpatch_StatsThrottle20070306EXT SWpatch_StatsThrottle20070306 manage 23 K 06 Mar 2007 - 23:45 UnknownUser Statistics page revised (8) with Throttling ability for netcams tested and working
SWpatch_StatsThrottle20070323EXT SWpatch_StatsThrottle20070323 manage 27 K 23 Mar 2007 - 00:52 UnknownUser Statistics page revised (9) with Throttling ability for netcams tested and working, and plain text output correct
SWpatch_StatsThrottle20070411EXT SWpatch_StatsThrottle20070411 manage 28 K 11 Apr 2007 - 22:39 UnknownUser Statistics page revised (10) with Throttling ability for netcams, and low_cpu implemented.
SWpatch_StatsThrottle20070928_svn223EXT SWpatch_StatsThrottle20070928_svn223 manage 29 K 09 Oct 2007 - 08:24 UnknownUser Statistics page revised (11) with Throttling ability for netcams, and low_cpu removed to be compatible with SVN223.
SWpatch_StatsThrottle20080117_for_svn305EXT SWpatch_StatsThrottle20080117_for_svn305 manage 34 K 19 Jan 2008 - 17:34 UnknownUser Patch to svn305 to show TCP buffers on Statistics pages. Some issues, see text.
fastserver.jpgjpg fastserver.jpg manage 33 K 13 Feb 2007 - 01:06 UnknownUser Stats page (7) output on fast server
motion-20060112-051001-2.diffdiff motion-20060112-051001-2.diff manage 3 K 13 Jan 2006 - 05:11 UnknownUser  
slowserver.jpgjpg slowserver.jpg manage 28 K 13 Feb 2007 - 01:06 UnknownUser Stats page (7) output on slow server
snapshot_load0.58.jpgjpg snapshot_load0.58.jpg manage 128 K 28 Feb 2007 - 01:03 UnknownUser Sempron 2400 system running 4 netcams, load 0.58 cpu 15% (throttling to 2fps)
snapshot_load1.08.jpgjpg snapshot_load1.08.jpg manage 128 K 28 Feb 2007 - 01:02 UnknownUser Sempron 2400 system running 4 netcams, load 1.08 cpu 50% (no throttling)
stats1.jpgjpg stats1.jpg manage 30 K 08 Feb 2007 - 04:03 AngelCarpintero  
throttle_320x240.jpgjpg throttle_320x240.jpg manage 61 K 21 Feb 2007 - 00:56 UnknownUser Throttling demo, 320x240
throttle_640x480.jpgjpg throttle_640x480.jpg manage 61 K 21 Feb 2007 - 00:56 UnknownUser Throttling demo, 640x480
throttle_lowcpu.jpgjpg throttle_lowcpu.jpg manage 68 K 11 Apr 2007 - 23:07 UnknownUser Screen capture of low_cpu=1 throtting a netcam
Topic revision: r50 - 22 Jul 2008, SimonW
Copyright © 1999-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Please do not email Kenneth for support questions (read why). Use the Support Requests page or join the Mailing List.
This website only use harmless session cookies. See Cookie Policy for details. By using this website you accept the use of these cookies.