Analog signal software filtering
- AndreyB
- Site Admin
- Posts: 14333
- Joined: Wed Aug 28, 2013 1:28 am
- Location: Jersey City
- Github Username: rusefillc
- Slack: Andrey B
Analog signal software filtering
https://sourceforge.net/p/rusefi/tickets/261/ Low-pass_filter for slow ADC sensors
Here is a data log - car not running, just cooling down in the garage. That's with 8 samples averaging 10 times a second
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/DataLogs/2016-01-28_21.27.58_not_running_8_samples_averaging.msl
Anyone to apply any voodoo math to propose LPF coefficients?
Here is a data log - car not running, just cooling down in the garage. That's with 8 samples averaging 10 times a second
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/DataLogs/2016-01-28_21.27.58_not_running_8_samples_averaging.msl
Anyone to apply any voodoo math to propose LPF coefficients?
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Re: Analog signal software filtering
Averaging is a low-pass filter, you can either increase the number of samples or reduce the sample rate. If you are average over 0.8secs then it should be fairly stable, its possible that you have a noise issue on the ADC.
Edit: Looking at that data I don't see what the issue is.
Edit: Looking at that data I don't see what the issue is.
Re: Analog signal software filtering
Was there a way with dev console to get just vbat and time stamp logged as fast as possible?
From the below picture we can see that you are dealing with +/- 3 ADC ticks. The ADC is updating once ever 100mS, but the log is capturing as fast as .155s and as slow as .172s, with a typical of about .16s. It would be handy if you could capture every ADC reading.
An exponential moving average (EMA) is a type of IIR filter that needs reasonably low CPU cycles and is reasonably easy to use. It only needs sample N and N-1, as well it only does a couple multiplications and addition(s). See below web page which includes an example snippet of code.
http://coactionos.com/embedded%20design%20tips/2013/10/04/Tips-An-Easy-to-Use-Digital-Filter/
I made a version of that in LibreOffice, saved as .xls. Below is the results of the filtering.
Source spreadsheet file attached. I'm not sure why I had to add the offset of -.325 in cell E2, there might be a bug in my spread sheet some where, or the offset might be constant. The green cell can be adjusted to change the response.
[edit]kb1gtt deleted the old graphics and old xls file which have updated copies below. [/edit]
From the below picture we can see that you are dealing with +/- 3 ADC ticks. The ADC is updating once ever 100mS, but the log is capturing as fast as .155s and as slow as .172s, with a typical of about .16s. It would be handy if you could capture every ADC reading.
An exponential moving average (EMA) is a type of IIR filter that needs reasonably low CPU cycles and is reasonably easy to use. It only needs sample N and N-1, as well it only does a couple multiplications and addition(s). See below web page which includes an example snippet of code.
http://coactionos.com/embedded%20design%20tips/2013/10/04/Tips-An-Easy-to-Use-Digital-Filter/
I made a version of that in LibreOffice, saved as .xls. Below is the results of the filtering.
Source spreadsheet file attached. I'm not sure why I had to add the offset of -.325 in cell E2, there might be a bug in my spread sheet some where, or the offset might be constant. The green cell can be adjusted to change the response.
[edit]kb1gtt deleted the old graphics and old xls file which have updated copies below. [/edit]
Last edited by kb1gtt on Sat Jan 30, 2016 3:19 am, edited 1 time in total.
Welcome to the friendlier side of internet crazy
Re: Analog signal software filtering
I just made an update to the above post, tuning the alpha for a faster response and adding the offset. I see the files were already viewed 3 times, so I suggest you folks update to these newest files.
Welcome to the friendlier side of internet crazy
- AndreyB
- Site Admin
- Posts: 14333
- Joined: Wed Aug 28, 2013 1:28 am
- Location: Jersey City
- Github Username: rusefillc
- Slack: Andrey B
Re: Analog signal software filtering
That log was taken TTL@14400, here's a much more detailed one taken via USB today:kb1gtt wrote: the log is capturing as fast as .155s and as slow as .172s, with a typical of about .16s. It would be handy if you could capture every ADC reading.
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/DataLogs/2016-01-29_08.29.41_not_running_8_samples_averaging.msl
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
- AndreyB
- Site Admin
- Posts: 14333
- Joined: Wed Aug 28, 2013 1:28 am
- Location: Jersey City
- Github Username: rusefillc
- Slack: Andrey B
Re: Analog signal software filtering
The magic is a bit broken because you are taking column P in but that's the wrong column.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Re: Analog signal software filtering
Found some other issues. Also the faster log is good, but the alpha's are wrong compared to what you need on the firmware. If you could log once every ADC reading, that would be perfect. Then your alpha could match your spread sheet. Below is what I get after the corrections.
- Attachments
-
- 2016-01-28_21.27.58_not_running_8_samples_averaging.xls
- (4.53 MiB) Downloaded 380 times
Welcome to the friendlier side of internet crazy
- AndreyB
- Site Admin
- Posts: 14333
- Joined: Wed Aug 28, 2013 1:28 am
- Location: Jersey City
- Github Username: rusefillc
- Slack: Andrey B
Re: Analog signal software filtering
Just implemented exp average, also bumped slow ADC speed to 20Hz. Logs are trying to be at 20Hz as well - not super precise but we do not have a choice
Alpha=1, then 0.5, then 0.333
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.06.58_8_average_20hz_alpha_1.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.08.27_average_20hz_alpha_0_5.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.09.15_average_20hz_alpha_0_333.msl
Can I declare this good enough? No offset in current implementation since offset scares me away. I guess initialization curve could be improved if S[-1] is declared equal to S[0] and not 0.
Alpha=1, then 0.5, then 0.333
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.06.58_8_average_20hz_alpha_1.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.08.27_average_20hz_alpha_0_5.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.09.15_average_20hz_alpha_0_333.msl
Can I declare this good enough? No offset in current implementation since offset scares me away. I guess initialization curve could be improved if S[-1] is declared equal to S[0] and not 0.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Re: Analog signal software filtering
I see the before and after look like this.
The below might be of interest, which would allow options for making this even smoother, as well it would allow different characteristics for signals like TPS or NO2.
http://www.iowahills.com/A7ExampleCodePage.html
They have source code published here
http://www.iowahills.com/IIR%20Algorithm/BilinearCode.txt
The constants are made from a .exe found here
http://www.iowahills.com/Downloads/Iowa%20Hills%20IIR%20Filters.zip
These are same scale, both are 20 seconds of capture. So sure it looks much smoother. However we can always do better The below might be of interest, which would allow options for making this even smoother, as well it would allow different characteristics for signals like TPS or NO2.
http://www.iowahills.com/A7ExampleCodePage.html
They have source code published here
http://www.iowahills.com/IIR%20Algorithm/BilinearCode.txt
The constants are made from a .exe found here
http://www.iowahills.com/Downloads/Iowa%20Hills%20IIR%20Filters.zip
Welcome to the friendlier side of internet crazy
Re: Analog signal software filtering
According to this page http://www.earlevel.com/main/2003/02/28/biquads/ biquad transposed direct form II are good IIR software filters for floating point maths. They use small CPU cycles especially if you have a FPU, which the STM32F4 has.
Example code about how to make the above is posed here http://www.earlevel.com/main/2012/11/26/biquad-c-source-code/ This is the magic part
Example code about how to make the above is posed here http://www.earlevel.com/main/2012/11/26/biquad-c-source-code/ This is the magic part
I made that into spreadsheet, and then used it to process the ADC readings from the log file. I got the a0, a1, --> b2 values from here http://www.earlevel.com/main/2013/10/13/biquad-calculator-v2/ and produced the below results.inline float Biquad::process(float in) {
double out = in * a0 + z1;
z1 = in * a1 + z2 - b1 * out;
z2 = in * a2 - b2 * out;
return out;
}
- Attachments
-
- Filters.xls
- (5.52 MiB) Downloaded 375 times
Last edited by kb1gtt on Sun Feb 07, 2016 11:20 pm, edited 1 time in total.
Welcome to the friendlier side of internet crazy
Re: Analog signal software filtering
Good to hear it's working reasonably well.
I figured out how to use the iowahills.com coefficients in the spread sheet. They A's and B's are flipped relative to the earlevel.com coefficients and the A0 is always 1, which is not normally used. See below picture which shows the values from the IIRFilters.exe entered into the spread sheet. Also note in this IIRFilters.exe if you specify more than 1 pole, the program will show you the coefficients if you cascade the filters. You can get better filtering by connecting the output of one filter to the input of the next filter. Updated spread sheet attached.
I figured out how to use the iowahills.com coefficients in the spread sheet. They A's and B's are flipped relative to the earlevel.com coefficients and the A0 is always 1, which is not normally used. See below picture which shows the values from the IIRFilters.exe entered into the spread sheet. Also note in this IIRFilters.exe if you specify more than 1 pole, the program will show you the coefficients if you cascade the filters. You can get better filtering by connecting the output of one filter to the input of the next filter. Updated spread sheet attached.
- Attachments
-
- Filters.xls
- (5.52 MiB) Downloaded 373 times
Welcome to the friendlier side of internet crazy
- AndreyB
- Site Admin
- Posts: 14333
- Joined: Wed Aug 28, 2013 1:28 am
- Location: Jersey City
- Github Username: rusefillc
- Slack: Andrey B
Re: Analog signal software filtering
I've finally tried BiQuad filtering - https://svn.code.sf.net/p/rusefi/code/trunk/firmware/controllers/math/biquad.cppkb1gtt wrote:I figured out how to use the iowahills.com coefficients in the spread sheet. They A's and B's are flipped relative to the earlevel.com coefficients and the A0 is always 1,
Mixed results.
On the one hand, CLT and IAT are pretty nice once I've implemented "initValue" method which sets initial z1 and z2, on the other hand I am considering TPS and AFT "slow" sensors so applying same filtering to those. It takes 20 seconds for the filter to transition from tps=0% to tps=100% while it takes me less than a second to press the pedal
- Attachments
-
- 2016-09-10_21.37.48.msl
- (313.41 KiB) Downloaded 765 times
-
- screenshot.png (25.54 KiB) Viewed 10439 times
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
Re: Analog signal software filtering
I believe we are sampling at 100 samples per second, is that correct? If so I'm expecting it takes about 8 seconds to come in from a fast and large step change. I wonder if you are some how doing it a 50 samples per second. These filters only know samples, so to get a cut off at a certain Hz, you need the samples per seconds to match. If you change your samples per second, you change your cut off frequency.
The above is for the magic numbers below.
However if we change to these magic numbers
We would have a step response like this.
That would also change the filtering to look more like this. For TPS this might still be a bit sluggish, but it might also be OK. If it's still sluggish, then we can change the magic numbers again.
At the end of the day, I don't know what the tolerable low pass is. It's a balancing act between throttle response and noise. If you can get rid of the noise, you can have a more aggressive low pass filter. We can change the magic numbers to change the low pass cut off knee, until we get a good balance. We can also cascade this biquad filer basically connecting the output of filter 1 to the input of filter 2, etc which makes for a sharper knee.
I would think this would be about as high of a cut off as might want to go with TPS. If you have a noisy TPS this would probably be a bit twitchy.
That's for these magic numbers
The above is for the magic numbers below.
However if we change to these magic numbers
We would have a step response like this.
That would also change the filtering to look more like this. For TPS this might still be a bit sluggish, but it might also be OK. If it's still sluggish, then we can change the magic numbers again.
At the end of the day, I don't know what the tolerable low pass is. It's a balancing act between throttle response and noise. If you can get rid of the noise, you can have a more aggressive low pass filter. We can change the magic numbers to change the low pass cut off knee, until we get a good balance. We can also cascade this biquad filer basically connecting the output of filter 1 to the input of filter 2, etc which makes for a sharper knee.
I would think this would be about as high of a cut off as might want to go with TPS. If you have a noisy TPS this would probably be a bit twitchy.
That's for these magic numbers
Welcome to the friendlier side of internet crazy
Re: Analog signal software filtering
For the purpose of breadcrumbs. This is a sweet day filter tool that was discussed
http://www.micromodeler.com/dsp/
http://www.micromodeler.com/dsp/
Welcome to the friendlier side of internet crazy