Analog signal software filtering

[AndreyB]

https://sourceforge.net/p/rusefi/tickets/261/ Low-pass_filter for slow ADC sensors

Here is a data log - car not running, just cooling down in the garage. That's with 8 samples averaging 10 times a second
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/DataLogs/2016-01-28_21.27.58_not_running_8_samples_averaging.msl
Anyone to apply any voodoo math to propose LPF coefficients?

[Rhinoman]

Averaging is a low-pass filter, you can either increase the number of samples or reduce the sample rate. If you are average over 0.8secs then it should be fairly stable, its possible that you have a noise issue on the ADC.
Edit: Looking at that data I don't see what the issue is.

[kb1gtt]

Was there a way with dev console to get just vbat and time stamp logged as fast as possible?

From the below picture we can see that you are dealing with +/- 3 ADC ticks. The ADC is updating once ever 100mS, but the log is capturing as fast as .155s and as slow as .172s, with a typical of about .16s. It would be handy if you could capture every ADC reading.

An exponential moving average (EMA) is a type of IIR filter that needs reasonably low CPU cycles and is reasonably easy to use. It only needs sample N and N-1, as well it only does a couple multiplications and addition(s). See below web page which includes an example snippet of code.
http://coactionos.com/embedded%20design%20tips/2013/10/04/Tips-An-Easy-to-Use-Digital-Filter/

I made a version of that in LibreOffice, saved as .xls. Below is the results of the filtering.

Source spreadsheet file attached. I'm not sure why I had to add the offset of -.325 in cell E2, there might be a bug in my spread sheet some where, or the offset might be constant. The green cell can be adjusted to change the response.

[edit]kb1gtt deleted the old graphics and old xls file which have updated copies below. [/edit]

[kb1gtt]

I just made an update to the above post, tuning the alpha for a faster response and adding the offset. I see the files were already viewed 3 times, so I suggest you folks update to these newest files.

[AndreyB]

the log is capturing as fast as .155s and as slow as .172s, with a typical of about .16s. It would be handy if you could capture every ADC reading.

That log was taken TTL@14400, here's a much more detailed one taken via USB today:

https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/DataLogs/2016-01-29_08.29.41_not_running_8_samples_averaging.msl

[AndreyB]

The magic is a bit broken because you are taking column P in but that's the wrong column.

[kb1gtt]

Found some other issues. Also the faster log is good, but the alpha's are wrong compared to what you need on the firmware. If you could log once every ADC reading, that would be perfect. Then your alpha could match your spread sheet. Below is what I get after the corrections.

vbat_filter.png (38.56 KiB) Viewed 11390 times

[AndreyB]

Just implemented exp average, also bumped slow ADC speed to 20Hz. Logs are trying to be at 20Hz as well - not super precise but we do not have a choice

Alpha=1, then 0.5, then 0.333

https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.06.58_8_average_20hz_alpha_1.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.08.27_average_20hz_alpha_0_5.msl
https://svn.code.sf.net/p/rusefi/code/trunk/misc/logs/2003_dodge_neon/2016-01-29_22.09.15_average_20hz_alpha_0_333.msl

Can I declare this good enough? No offset in current implementation since offset scares me away. I guess initialization curve could be improved if S[-1] is declared equal to S[0] and not 0.

[kb1gtt]

I see the before and after look like this.

Vbat_20s_alpha1.PNG (39.64 KiB) Viewed 11382 times

Vbat_20s_alpha.333.PNG (23.35 KiB) Viewed 11382 times

These are same scale, both are 20 seconds of capture. So sure it looks much smoother. However we can always do better

The below might be of interest, which would allow options for making this even smoother, as well it would allow different characteristics for signals like TPS or NO2.

http://www.iowahills.com/A7ExampleCodePage.html

They have source code published here

http://www.iowahills.com/IIR%20Algorithm/BilinearCode.txt

The constants are made from a .exe found here

http://www.iowahills.com/Downloads/Iowa%20Hills%20IIR%20Filters.zip

[kb1gtt]

According to this page http://www.earlevel.com/main/2003/02/28/biquads/ biquad transposed direct form II are good IIR software filters for floating point maths. They use small CPU cycles especially if you have a FPU, which the STM32F4 has.

Example code about how to make the above is posed here http://www.earlevel.com/main/2012/11/26/biquad-c-source-code/ This is the magic part

inline float Biquad::process(float in) {
double out = in * a0 + z1;
z1 = in * a1 + z2 - b1 * out;
z2 = in * a2 - b2 * out;
return out;
}

I made that into spreadsheet, and then used it to process the ADC readings from the log file. I got the a0, a1, --> b2 values from here http://www.earlevel.com/main/2013/10/13/biquad-calculator-v2/ and produced the below results.

Vbat_BiQuad_LPF.PNG (21.58 KiB) Viewed 11370 times

[kb1gtt]

Good to hear it's working reasonably well.

I figured out how to use the iowahills.com coefficients in the spread sheet. They A's and B's are flipped relative to the earlevel.com coefficients and the A0 is always 1, which is not normally used. See below picture which shows the values from the IIRFilters.exe entered into the spread sheet. Also note in this IIRFilters.exe if you specify more than 1 pole, the program will show you the coefficients if you cascade the filters. You can get better filtering by connecting the output of one filter to the input of the next filter.

Filters_with_iowahills.com_coeffs.PNG (67.46 KiB) Viewed 11352 times

Updated spread sheet attached.

[AndreyB]

I figured out how to use the iowahills.com coefficients in the spread sheet. They A's and B's are flipped relative to the earlevel.com coefficients and the A0 is always 1,

I've finally tried BiQuad filtering - https://svn.code.sf.net/p/rusefi/code/trunk/firmware/controllers/math/biquad.cpp
Mixed results.

On the one hand, CLT and IAT are pretty nice once I've implemented "initValue" method which sets initial z1 and z2, on the other hand I am considering TPS and AFT "slow" sensors so applying same filtering to those. It takes 20 seconds for the filter to transition from tps=0% to tps=100% while it takes me less than a second to press the pedal

[kb1gtt]

I believe we are sampling at 100 samples per second, is that correct? If so I'm expecting it takes about 8 seconds to come in from a fast and large step change. I wonder if you are some how doing it a 50 samples per second. These filters only know samples, so to get a cut off at a certain Hz, you need the samples per seconds to match. If you change your samples per second, you change your cut off frequency.

IIR_LPF_36.9samples_response.png (11.34 KiB) Viewed 10428 times

The above is for the magic numbers below.

IIR_LPF_3.69samples_100freq.png (28.42 KiB) Viewed 10428 times

However if we change to these magic numbers

IIR_LPF_0.360samples_100freq.png (31.85 KiB) Viewed 10428 times

We would have a step response like this.

IIR_LPF_3.60samples_response.png (13.83 KiB) Viewed 10428 times

That would also change the filtering to look more like this. For TPS this might still be a bit sluggish, but it might also be OK. If it's still sluggish, then we can change the magic numbers again.

IIR_LPF_3.60samples_Smoothed.png (21.28 KiB) Viewed 10428 times

At the end of the day, I don't know what the tolerable low pass is. It's a balancing act between throttle response and noise. If you can get rid of the noise, you can have a more aggressive low pass filter. We can change the magic numbers to change the low pass cut off knee, until we get a good balance. We can also cascade this biquad filer basically connecting the output of filter 1 to the input of filter 2, etc which makes for a sharper knee.

I would think this would be about as high of a cut off as might want to go with TPS. If you have a noisy TPS this would probably be a bit twitchy.

IIR_LPF_0.180samples_response.png (13.8 KiB) Viewed 10428 times

That's for these magic numbers

IIR_LPF_0.180samples_100freq.png (31.34 KiB) Viewed 10428 times

[kb1gtt]

For the purpose of breadcrumbs. This is a sweet day filter tool that was discussed

http://www.micromodeler.com/dsp/