[rusEfi] Miata NA 1990, 1.6L stock #21

[AndreyB]

Ops, you are right - we've seen it on both channels, no point in swapping channels.

Just made error codes unique and added lastErrorCode into all three log files, build server still building bundle, that would be firmware version #10199.

[kb1gtt]

Swapping pins was a long shot, and mostly to double check it's not a hardware problem. MOSFET's can be easily triggered, you can get stray voltages if you happen to have energy coupling between traces that are next to each other. This kind of issue is more noticeable with analog inputs which are high impedance and you are often looking at very small differences in voltage. In digital land you can often see a bump in the signal when a near by pin is toggled, or when similar things like that happen. Some times that energy can couple across a port in the MCU this happens when you over drive a pin, with a soft power supply. Basically when the port fires if you don't have proper bulk storage it can pull the power supply line down for a short period of time on that port and this energy can ripple to other pins on that port. This energy can also couple across the traces in the PCB layout. In this case it would have been a long shot because we have resistors like R408 which lowered the impedance at the MOSFET which decreases the chances of cross coupling in the traces, as well it's high enough resistance that the bulk storage could not be over drawn. As well the PCB using that 4 layer with the GND plane floods pretty much everywhere. This also helps prevent the cross coupling issues. However if there was a stay cold solder joint, which disconnected R408, you could identify that kind of hardware problem by switching to a different channel. There may have been some concern that switching the pin would help identify if there was some other bit of code that was triggering this pin in hardware. As well those are things that you guys in the field can check, instead of relying on waiting for @russian to track down a firmware bug. What ever the problem is, it's not supper obvious and not currently known where it is being sourced from.

However having seen this on another pin, and I seem to recall it was also seen on a second setup, this really points to a firmware bug not a hardware issue. Switching to another pin, might still be worth while just for the sake of checking. The source of the problem is not yet know, and we are mostly relying on gut feel, so checking hardware doesn't hurt, as it could be the lottery winner at this point. However we suspect that the lottery will be given credit to the firmware.

[puff]

oops, initially i didnot realized both channels were affected. besides, i didn't imply that was to rule out some hardware fault.
what I meant to say is do you have any collision checkups in firmware?
as far as i got, it loads configs from some area in memory. could it be so that the same physical pin is somehow mapped to various parts of code (so that you change the pin state in two different functions)?

[AndreyB]

Tom has emailed me the new logs, no fuel logic codes there. Only one syncError warning code which I suspect is part of firmware initialization, I will double-check and see if I can eliminate or ignore that warning since that's a false warning.

[AndreyB]

Code: Select all

		if (rpm > 2 * engineConfiguration->cranking.rpm) {
			const char *outputName = event->output->name;
			if (prevOutputName == outputName) {
				warning(CUSTOM_OBD_46, "looks like skipped fuel event");
			}
			prevOutputName = outputName;
		}

Not a fix but I am expecting this logic to throw a code every time there is a fuel miss. revision #10233

I am curious what's different between idle and 3k, I wonder if the issue is caused by some combination of trigger offset and fuel amount which only happens on the particular tune around 3K? just a theory, planning to write some code to self-stimulate with RPMs around 3K and more or less realistic MAP values in order to reproduce the issue on a bench. Maybe will just start with a potentiometer

[tomiata]

Thanks, Russian. This new version does capture when misses happen! I drove a couple miles and tried to watch TS gauges monitoring the warn count. It doesn't increment the count for every time that I feel a hesitation but does catch it in the act.

This video captures the rus136.msl run below.

[video][/video]

[AndreyB]

(slowly getting back from a mini-vacation in DC)

Thank you @ for bench steps to reproduce issue https://sourceforge.net/p/rusefi/tickets/299/

Code: Select all

set_engine_type 41
enable self_stimulation
rpm 2900
disable sd
writeconfig
(play with a potentiometer connected to GND<>PA7<>VCC)

[AndreyB]

I've noticed that disconnecting potentiometer always causes warningCounter to increase (as long as after 10 seconds which was the hard-coded warning period). Made changes to make warningPeriod configurable via "get warningPeriod" and "set warningPeriod XXX" commands and yes, definitely every time I connect/disconnect the pot I get the warningCounter increment which tells me MAP value change is to blame which make me wonder about acceleration logic playing a role.

I was hoping that turning off acceleration set_engine_load_accel_multiplier 0 would cure the fuel miss issue but that does not seem to be the case, to be continued.

[tomiata]

I'm not familiar with all the magic calculations going on to calculate fuel and timing, but looking at the conditions when it happens I made this plot.

Fuel-miss-graph.jpg (184.82 KiB) Viewed 17558 times

This may be a clue. It looks like a miss can occur when their is large change in MAP value, which corresponds to a large change in advance.

Is the bug showing up only in Mazda engines? What about the 1.8L Mazda?
Are they all using Speed Density?
Are we all the 1.6L Miatas using the same or similar timing tables?
I'm using the timing table from @thommm, and have not tweaked it at all.

[AndreyB]

This may be a clue. It looks like a miss can occur when their is large change in MAP value, which corresponds to a large change in advance.

Also large change in fuel which is probably more important here.

Made some progress on this issue today. I am starting to think that there are potentially multiple issues here
1) part of the issue seems in fact be related to acceleration enrichment: default enrichment settings are putting injectors above 100% duty cycle which is not handled properly. Just added a check for above 100% with a new warning message but have not yet improved handling of this situation. Next road test should definitely be with all accelerations disabled. On the other hand, we have this issue even while cruising right?

2) I am starting to think that current 'skipped fuel event' logic is prone to false-positives - I am suspecting that a transition from below 50% duty cycle to above 50% duty cycle does in fact look like a skipped fuel event.

Getting late here, to be continued.

[AndreyB]

PS: rus64.msl from http://rusefi.com/forum/viewtopic.php?f=3&t=660&start=60#p19456 has max duty cycle of 48% in spikes but on the other hand the logic analyzer shots are nowhere near 50% and does not really look like any chance in fuel or advance. I am a bit lost.

[Thommm]

I have made a little progress with the misfires, I noticed a lot of improvement after adding a low pass filter after the map sensor. I decoupled it properly right at the output (5V referenced), but not after the divider to get it at 2.5V max. The divider is made up out of two 39k ohm resistors, after adding a 82pF cap there misfires weren't so abundant with a high acceleration multiplier anymore. In the logs the map trace appeared slightly less noisy. The filter has a cutoff at 40kHz, so that shouldn't impact reaction time too much. If it's unclear I'll draw up a schematic.

[AndreyB]

I have made a little progress with the misfires, I noticed a lot of improvement after adding a low pass filter after the map sensor.

But why? I am relatively sure there are software issues, how would a hardware change help?

[Thommm]

Oh it's not our main issue, that's for sure. I just noticed in the log that acceleration enrichment spiked quite a lot, this fixed it. Could help for some people with noisy signals. But indeed, not our software fix By the way, I'm leaving tomorrow on holiday to France, RusEFI is going to be making around 2000km's (1500ish miles) in 7 days

[kb1gtt]

I think I see what he is saying

If you have some small noise on the TPS, it could potentially trigger very short fuel trim enrichment(s), which may also clear as fast as they arrived and occasionally cause these skipped fuel pulses. By adding a hardware filter, it may have cleaned up the noise, and removed these short term spikes. A long term solution would likely be to make the algorithm not react to such short noise spikes. I would guess that this is being seen on these Miata's as there is some common wiring issue that allow more noise to couple through at around 3kRPM.

[AndreyB]

But do we have the data to support this hardware change? We have misses even while cruising and both load and TPS enrichments have pretty high thresholds so that these are not activated on smaller changes in sensors.

[kb1gtt]

Noise will look like high rates of change, even though the TPS is constant. I do not think we have data to support this change. It sounds like this fellow made an observation and it may be a patch to the problem. I seem to recall a simulation where captured data is sent to a discovery board. Can we modify that captured data, adding noise to the TPS and see if that triggers the skipped pulses? If we can do that it would likely provide the data to back this up.

[AndreyB]

Still looking into this, actually made some small progress: I've made a custom firmware to artificially oscillate MAP reading in the 28 to 37 kPa range and I am now having a miss four times a second. Getting closer to a fix.

[Thommm]

Sounds good, it was the most annoying thing when I was in France I've also got another suggestion: long/short therm correction of the fuel according to the wideband. After getting back from the holiday my map changed only slightly (5% max or something) but that had a huge impact on drivability. While driving in mountainous terrain instead of my normal flat (Dutch) landscape I had frequent misfires, short therm correction would have improved that a lot I think. Is there a suggestions topic somewhere?

[AndreyB]

another suggestion: long/short therm correction of the fuel according to the wideband.

http://rusefi.com/forum/viewtopic.php?f=5&t=1046

[AndreyB]

https://sourceforge.net/p/rusefi/tickets/299/ looks fixed on the bench, how about on your car?

[tomiata]

https://sourceforge.net/p/rusefi/tickets/299/ looks fixed on the bench, how about on your car?

Thanks for the work on this. I did a brief test tonight and drove about 10 miles. Cruising at 60mph around 3000 rpm is smooth, but accelerating through gears still has some hesitation. I see the warn count status corresponds to this observation. Here is a log. I'll do more testing tomorrow.

[AndreyB]

So good news we have progress

Re-opening https://sourceforge.net/p/rusefi/tickets/299/ since it looks there is another issue there. Next step reproducing this on a bench

I've noticed your MAP value is often above 100kPa and often in the 120s, that's not right for a naturally aspirated.

Another note for myself is to improve the column names in the rusEfi log files.

[tomiata]

Yes definitely, progress is good news!

Any suggestions on my MAP? I have an mpx4250 soldered to the board inside the enclosure and a vacuum line running from the intake manifold near the throttle body. The AFM is still in place. I didn't remove it or mechanically disable it. And I have an aftermarket cone intake filter.

On other progress I got my TPS working last week and should be sane in the logs now. It was a wiring mistake on my part.

[tomiata]

I did more testing and gathered some data with a logic analyzer. Idle is smooth, cruising is smooth. Shifting through gears is rough with miss/hesitations, and accelerating from 60mph to 70mph in 5th gear is rough also.

I gathered samples from the analyzer while driving (with the assistance of my son running the laptop while I'm driving.)
I have samples of inj miss and spark miss, and combinations. Here are few screen shots.

inj-miss1.jpg (104.41 KiB) Viewed 17380 times

inj-miss2.jpg (101.52 KiB) Viewed 17380 times

inj-spark-miss.jpg (101.19 KiB) Viewed 17380 times

I wonder if some of this is problems with my tuning? Here are all the logs I saved and my tune. If anyone has suggestions on other tests to try let me know. THANKS!

https://drive.google.com/open?id=0By8Qyq7Eklg-WU9qNjhPUWFZdVk

[Dial0]

Is there a way to confirm that all the events in the fuel schedule have been processed before you switch tables?

Maybe the fuel and spark schedule should be concurrent queue data types?

If we assume a 4 cylinder engine, every engine cycle 4 new spark event and 4 new fuel events will be loaded into a fuel queue and a spark queue, and when they are loaded into the timers (or however they are consumed) they are popped off the queue?

That way we know there is no data being lost when there is a switch.


Edit: A quick fix might be that when the last event in the schedule is processed, it then calls the function that switches table pointers?

[Dial0]

In reply to Tom,

If there are output events missing, its almost certain to be a hardware/firmware issue.
If the tune was incorrect, the outputs would still happen, just the engine would not respond well to those outputs (spark at the wrong time, not enough duty cycle on the fuel injector causing a missfire).

[AndreyB]

I have samples of inj miss and spark miss, and combinations.

Great stuff and really helpful pictures! So the issue is not specific to just fuel - interesting. Fuel and spark use slightly different algorithms for historical reason and it's interesting that both are affected.

Maybe the fuel and spark schedule should be concurrent queue data types?

Please dive into the code & be more specific

If we assume a 4 cylinder engine, every engine cycle 4 new spark event and 4 new fuel events will be loaded into a fuel queue and a spark queue, and when they are loaded into the timers (or however they are consumed) they are popped off the queue?

I could be wrong and I could be over-complicating things, but I am not scheduling using just 'time since start of cycle' - I am scheduling everything based of the closes trigger event. This case I am hopefully getting better precision in case of rapidly changing RPM, but all that makes the code kind of tricky.

A quick fix might be that when the last event in the schedule is processed, it then calls the function that switches table pointers?

The challenge is that with 60/2 triggers I cannot afford to have too much logic in the trigger interrupt handler - I need to do scheduling on a different thread in order to have 60/2 performance.

My next step would be to reproduce this on a bench.

[Dial0]

ahh I think I understand more so what is going on now.

With the two tables you are trying to make sure the ISR isn't reading the same table the fuel calc might be modifying. Incase the ISR is called in the middle of the fuel calc function?

Where is your code is the scheduler ISR?

I am assuming that it runs at fixed rate, 100khz or something?
It then looks at the fuel schedule and checks if any pins need to change state?

Or is the scheduler ISR time set to the next event each time?

for example if there was an injectionend event and the ISR runs, changes the pin to Low, then reschedules the ISR to run at the next event which is a spark event.
Then before the ISR runs again the schedule table is updated.
The ISR then runs and the event it was scheduled for has then shifted, either slightly sooner or slightly later.

If slightly sooner it needs to change the corresponding pin state immediately and if slightly later does it reschedule to the slightly later time or change the pin state anyway?


When you say scheduling off the closest event, does that mean you are constantly recalculating the event timing?
So the fuel scheduler is running more than once per engine cycle?

this is what I am imagining the code for the fuel scheduler to work like, with UpdateActiveTable(&active) updating the entire table, not just events that haven't happened yet.

Code: Select all

SudoCode:

CopyActiveTabletoWorking(&active,&working)

DofuelCalculations(&working)

SetactivetableMutex()
UpdateActiveTable(&active)
UnsetActiveTableMutex()

[tomiata]

Looking at the saved traces, I found other odd case. After an inj miss, the other injector has an odd cycle and overlaps with the first.

two-at-once.jpg (105 KiB) Viewed 17339 times

Also, another potential clue: I only see injector miss on inj1 and spark miss on spark3.