[rusEfi] Miata NA 1990, 1.6L stock #21

Your chance to introduce yourself and your vehicle
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

Thanks for the report, much appreciated. Also thanks for posting the issue in the forum. We look to let others know about these issues so they might chime in and perhaps add different perspectives and different sources of information. Getting the information and reports is more important than worries about developer pride.

I'm looking to see if there are potential hardware issues. Perhaps a brown out condition happened during cranking, adding a small blip which may have caused some wonky memory. Perhaps something needs to be blocked during bootup. What ever caused the issue is currently an un-known. I'm not sure how we might be able to reproduce the situation. If you have any suggestions on how we might be able to reproduce the issues that would be great. It would be great if we can get our eyes on it in a development environment.
Welcome to the friendlier side of internet crazy :)
RustyGargoyle
contributor
contributor
Posts: 55
Joined: Sat Dec 03, 2016 2:57 am

Re: Miata NA 1990, 1.6L stock #21

Post by RustyGargoyle »

kb1gtt wrote: If you have any suggestions on how we might be able to reproduce the issues that would be great.
Bench testing the discovery board or better yet a full franken board, not sure which version he has. Run it off a couple of batteries to get the internal resistance in series and boot the board then slap on a fat load. should cause it to brown out again and potentially recreate the situation.

No idea how you would trace/debug that in the MCU thou.

Maybe if it relied on an external eeprom for config you can use a logic analyzer. But I dont think thats the case.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

tomiata wrote:I found one glitch at a fuel stop. I usually switch on and wait for the O2 sensor to warm up, but this time switched on and tried to start very quickly. It failed to start and I found that the board had lost the engine type setting. It might due to 12v supply going low when the board is installing.
But this was also on a warm engine after highway driving, so your battery was as happy as it would ever be, and your engine was as easy to crank as it would ever be?

I recall something similar maybe twice over the cource of few years. There is a ticket for something similar - https://sourceforge.net/p/rusefi/tickets/186/

On start up the firmware read the configuration and runs a CRC validation, if CRC is failed configuration is reset to default for this engine - again, based on engine type recovered from (possibly damaged) flash configuration.

Code: Select all

persisted_configuration_state_e readConfiguration(Logging * logger) {
	flashRead(FLASH_ADDR, (char *) &persistentState, PERSISTENT_SIZE);

	persisted_configuration_state_e result;
	if (!isValidCrc(&persistentState)) {
		result = CRC_FAILED;
		resetConfigurationExt(logger, DEFAULT_ENGINE_TYPE PASS_ENGINE_PARAMETER);
	} else if (persistentState.version != FLASH_DATA_VERSION || persistentState.size != PERSISTENT_SIZE) {
		result = INCOMPATIBLE_VERSION;
		resetConfigurationExt(logger, engineConfiguration->engineType PASS_ENGINE_PARAMETER);
	} else {
		/**
		 * At this point we know that CRC and version number is what we expect. Safe to assume it's a valid configuration.
		 */
		result = OK;
		applyNonPersistentConfiguration(logger PASS_ENGINE_PARAMETER);
	}
	// we can only change the state after the CRC check
	engineConfiguration->firmwareVersion = getRusEfiVersion();
	return result;
}
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

I can add errorCode for CRC failure but that would only be useful if connected to firmware right after CRC fail, or if SD logs would have it. It would probably also depend on reducing the error code noise on firmware restart, I assume the value is never zero as of now.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

Being stuck on a road is quite depressing so https://sourceforge.net/p/rusefi/tickets/186/ is the new highest priority defect unfortunatelly.

new error code: CUSTOM_ERR_FLASH_CRC_FAILED = 6141

And we need steps to reproduce. Good news is I have six Miata starters and I only need 3.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
puff
contributor
contributor
Posts: 2961
Joined: Mon Nov 11, 2013 11:28 am
Location: Moskau

Re: Miata NA 1990, 1.6L stock #21

Post by puff »

I do recall at least couple of times when the firmware suddenly forgot the settings, on bare discovery board.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

puff wrote:I do recall at least couple of times when the firmware suddenly forgot the settings, on bare discovery board.
steps to reproduce? :)
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

Tom, was TunerStudio connected during the incident?
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
puff
contributor
contributor
Posts: 2961
Joined: Mon Nov 11, 2013 11:28 am
Location: Moskau

Re: Miata NA 1990, 1.6L stock #21

Post by puff »

If I knew of such, I would have filed a bug report ;-)
I would say this happens either at startup or at shutdown. And of course I learn about that firmware reset when connecting the dev console.
RustyGargoyle
contributor
contributor
Posts: 55
Joined: Sat Dec 03, 2016 2:57 am

Re: Miata NA 1990, 1.6L stock #21

Post by RustyGargoyle »

Is it possible to just have the mcu try to compute a new crc or reload the configuration if it failed? or maybe a hardware reset from software.
Perhaps external memory is needed to store these base configs.
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

russian wrote:Tom, was TunerStudio connected during the incident?
Yes, TS was connected. Engine was warm. I had stopped to check oil etc. Might have been switched off for 10 minutes.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

tomiata wrote:Yes, TS was connected.
was auto-tune running? An easy explanation would be firmware writing to flash while you turn the key off so that flash memory was half-written. I will add some fkash footprint revision number to the LCD screen so that it would be easy to see when flash is updated and when it is not.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

russian wrote:
tomiata wrote:Yes, TS was connected.
was auto-tune running? An easy explanation would be firmware writing to flash while you turn the key off so that flash memory was half-written. I will add some fkash footprint revision number to the LCD screen so that it would be easy to see when flash is updated and when it is not.
Sorry, change that. Laptop was connected, but should have been suspended, asleep since I stopped and run the laptop off of an inverter. Auto-tune was NOT running.
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

Some thoughts. Perhaps as a patch until we are confident that this is all sorted out, we should include a back up copy of the parameters, then allow restoring the back up parameters via joy stick. At work I commonly have to do this in temperature controllers, or in PLC programs such that if the customer mangles some parameters they can do a factory reset and get it operational again.

I also wonder, could we have the ECU shut down if we see the 12V goes away? We have a bunch of caps, on the 5V, so we'll continue to operate for some amount of time after the 12V has gone away. Perhaps a high priority thread could check that 12V signals is above 6V and if it's below, then safely exit any flash writing operations and shut down. I've seen this done with a resistor divider and IRQ. Basically if the IO pin drops below the threshold, the pin toggles which triggers the high priority code.
Welcome to the friendlier side of internet crazy :)
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

I tried reproducing the conditions of losing the engine type a few times, key on quickly and cranking with the laptop connected.

Here is what 12v looks like at Frankeso, laptop is plugged in but suspended.
rusefi-12v-key-on-crank1.jpg
rusefi-12v-key-on-crank1.jpg (24.81 KiB) Viewed 16680 times
rusefi-12v-key-on-crank2.jpg
rusefi-12v-key-on-crank2.jpg (24.71 KiB) Viewed 16680 times
Here is 12v again with laptop powered on.
rusefi-12v-key-on-crank-wlaptop.jpg
rusefi-12v-key-on-crank-wlaptop.jpg (21.9 KiB) Viewed 16680 times
Here is 5v on Discovery, looks constant for all times that I tried it cranking quickly or just switching on.
rusefi-5v-keyon-crank1.jpg
rusefi-5v-keyon-crank1.jpg (24.67 KiB) Viewed 16680 times
Does not look like Discovery 5v supply is affected. Maybe it was just a random event?

Keeping a backup of the config data structure sounds good. If you CRC the structure for each update you could auto-recover if one copy is bad.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

My fear is this is not random, just low-probability. Having two version of config in flash is totally an option, but until we can reproduce the issue changing anything in the code is probably too soon :(

Detecting 12v off event would not help since flash write suspends execution for two seconds any way - so the bad scenario is starting to write flash and than loss of +12v which we would not react to since no execution anyway.

I guess the fact that this is really not often kind of helps me feel better. The yellow car for instance did not report this ever. Let me ask @
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

How about a test rig set up with discovery that turns on power and watches for serial data to confirm config is sane? It could run a few seconds then repeat and stop if the config changed. I could probably get my son set that up since he is off school for Christmas.
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

Can you capture the 5V signal at a faster sweep rate? Once the input and output caps are fully charged I expect the power supply can survive a 4V for 40mS voltage sag. However as those caps are charging up, perhaps there is a dynamic condition that's causes a small blip. I recall the official LM2596 has some hysteresis, which allows the input cap to charge a bit before the output starts to consume energy from the input caps. However who knows what these Chinese chips may have removed from the ASIC. If that hysteresis has been removed, then I could see a potential brown out during the dynamic 0V to 5V transition.

However during this short window if time, I expect that the STM32 is not yet programming the flash, so I doubt it's an issue, but I figure we should take a look and see if possible.

My guess is that it was turned off while flash was being written to. Do you recall if you had TS / auto tune running when you turned the key off?
Welcome to the friendlier side of internet crazy :)
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

kb1gtt wrote:Do you recall if you had TS / auto tune running when you turned the key off?
The answer above was NO.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

OK so no TS when it was working and shut down, and no TS when it was booted and no worky. What writes to the parameters, other than TS? About the only way I can think of the flash getting corrupted is if it's being written too, and then you loose power. I seem to recall the memory is done in blocks, such that it first erases the block, then writes the new data. If it dies while that block has not been re-written, it gets corrupted. I've heard of low voltage causing bits to go bad, but that is protected with brown out protection. I'm getting back to thinking that something has to be wrong with BOR settings.
Welcome to the friendlier side of internet crazy :)
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

kb1gtt wrote:Can you capture the 5V signal at a faster sweep rate?
Here is 5v turn on at 100us, 1ms, and 100ms per division:
frankenso-5v-pwron-100us.jpg
frankenso-5v-pwron-100us.jpg (20.51 KiB) Viewed 16659 times
frankenso-5v-pwron-1ms.jpg
frankenso-5v-pwron-1ms.jpg (20.14 KiB) Viewed 16659 times
frankenso-5v-pwron-100ms.jpg
frankenso-5v-pwron-100ms.jpg (19.9 KiB) Viewed 16659 times
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

tomiata wrote:How about a test rig set up with discovery that turns on power and watches for serial data to confirm config is sane? It could run a few seconds then repeat and stop if the config changed. I could probably get my son set that up since he is off school for Christmas.
Worth a try!
As is firmware uses only binary protocol now. Please download latest version so that you have disable binary_mode_console command

Code: Select all

set_engine_type 41
disable binary_mode_console
writeconfig
if thigs are well there will be MIATA_1_6 message every 10 or so seconds. If firmware resets MIATA would dissaper I guess and it would say CUSTOM or AUDI or else.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

This is looking good. I see at 0.2mS it's charged the caps and such to about 2V. This is about when I expect the STM32's caps and such are starting to get charged and the STM32 XTAL is starting. You can see this new load as a slight warping of the ramp up voltage. Then at 0.6mS you see a peak of 5.5V, which is he hysteresis I was talking about. This slight overshoot is to allow the upstream caps to get good and charge before the downstream loads are pulling from this bulk energy storage cap. The time between 0.2mS and about 0.3mS is where I wonder about the possibility of short term dynamic issues.

Per page 83 of this datasheet http://www.st.com/content/ccc/resource/technical/document/datasheet/ef/92/76/6d/bb/c2/4f/f7/DM00037051.pdf/files/DM00037051.pdf/jcr:content/translations/en.DM00037051.pdf

It notes VPOR/PDR for rising edge is between 1.64 and 1.80V. We should be good when above 1.7V, but the VPOR technically allows as low as 1.64V. In this areas it's a possible chance that the voltage is too low to allow for 100% accurate bit error rates. So some funky stuff could potentially happen. It also notes that if we activated VBOR1 it would bump that up to 2.23 to 2.33V, which would remove all possibilities of issues at these low voltages.

I don't know if this is what happened or not. I would really like to find a way to reproduce the issue, such that we can test before and after and verify that it has been squashed. Either way, I think it's a good idea to activate VBOR1.
Welcome to the friendlier side of internet crazy :)
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

russian wrote:
tomiata wrote:How about a test rig set up with discovery that turns on power and watches for serial data to confirm config is sane? It could run a few seconds then repeat and stop if the config changed. I could probably get my son set that up since he is off school for Christmas.
Worth a try!
As is firmware uses only binary protocol now. Please download latest version so that you have disable binary_mode_console command

Code: Select all

set_engine_type 41
disable binary_mode_console
writeconfig
if thigs are well there will be MIATA_1_6 message every 10 or so seconds. If firmware resets MIATA would dissaper I guess and it would say CUSTOM or AUDI or else.
In setting up the test rig, I zapped my spare Discovery board, was not cafeful, don't ask. And I bought 2 more from Digikey.
I loaded up rusefi and get output like I expect when powered from the USB, but when I try to power from just the 5V pin on the discovery header, it looks like rusefi does not start. LD1 led flashes, no blinking blue LD6. Is there something different about new Discovery boards? Or is there some firmware that needs to be wiped? This one says STM32F407G-DISC1.
Attachments
STM32F407G.jpg
STM32F407G.jpg (994.4 KiB) Viewed 16546 times
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

This looks to be http://rusefi.com/forum/viewtopic.php?f=4&t=1027

Long story short - you need to update st-link firmware (ST-Link Utility->Update firmware), and that seem to fix the issue
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
tomiata
contributor
contributor
Posts: 234
Joined: Sat Aug 02, 2014 2:30 am
Location: Texas

Re: Miata NA 1990, 1.6L stock #21

Post by tomiata »

russian wrote:This looks to be http://rusefi.com/forum/viewtopic.php?f=4&t=1027

Long story short - you need to update st-link firmware (ST-Link Utility->Update firmware), and that seem to fix the issue
I got it working and ran it through 7600 power cycles will no problems. This was using an arduino to control a relay to switch 5v on/off to the board, and I captured the serial output separately. It ran just 6 seconds and powered off. The string "Emulating MIATA_NA_1_6" came out every time. I didn't really expect it to fail, but I think it was useful to check this.

I'll try running a test with my frankenso board from the car next.

So the Rev D. boards only need a firmware update to be able to run rusefi? The current version of the ST-Link software is 4.0, and that worked to get rusefi to start up. But, I could not get the micro usb port (the one next to the audio plug) on this Rev. D board to work. It did not show up on the pc and I configured the board with the java console using the alternate PC10/PC11 serial pins. Are some other mods to the board required for that to work?
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

After the update, did you wait like 15 minutes for the new driver to install on the PC? It's kind of annoying, but it sits idle while it times out on a windows driver check.
Welcome to the friendlier side of internet crazy :)
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

tomiata wrote:So the Rev D. boards only need a firmware update to be able to run rusefi?
Yes. Still unclear if boards even have any hardware changes - could be that next generation ST Link firmware is the only difference.

As Jared says, installing the driver sometimes takes forever. Also sometimes it does not work well actually - I recall last time with Windows 10 was an issue for me and somehow I've located https://svn.code.sf.net/p/rusefi/code/misc/install/drivers/chibios_virtual_com_port_driver_oem112.zip driver which I believe have helped me.

Sorry for the slow response.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Miata NA 1990, 1.6L stock #21

Post by AndreyB »

As next step of endurance testing can we try adding

set_int 2472 100
writeconfig
set_int 2472 101
writeconfig
set_int 2472 102
writeconfig

And adding power off with a random delay after writeconfig?

set_int offset value command would modify alternator output frequency which is a safe value to adjust - offset 2472, random valid values are 100, 101 and 102.

writeconfig would initiate flash burn which should take about 2 seconds

I am curios if shutting power off too soon would kill the firmware or at least the miata configuration.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
kb1gtt
contributor
contributor
Posts: 3758
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: Miata NA 1990, 1.6L stock #21

Post by kb1gtt »

russian wrote:And adding power off with a random delay after writeconfig?
Could this be not random, but each time increase it by a small amount and do it from a min amount of time and keep doing it up to a max amount of time, such that we have a range, with a sweep in between? What is the min and max time that we would expect to be a potential problem? Also could that sweep be done like 10 or 100 times? It's possible this issue only happens at certain times, then only happens like 1 out of 10 times.
Welcome to the friendlier side of internet crazy :)
Post Reply