EMStudio and rusEfi

It's all about the code!
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

EMStudio and rusEfi

Post by AndreyB »

There is https://github.com/malcom2073/emstudio and there is https://github.com/malcom2073/emstune - not exactly sure what's the relationship between the two, emstune seems to have more recent activity.

I know some time ago there were attempts to teach EMStudio TunerStudio protocol which would allow at least three ECU (inluding rusEfi and speeduino) to be tuned by that tool. Not sure where it is right now but technically could end up being a true open source tuner.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Hey!

A bit of overview:
EMStudio
https://github.com/malcom2073/emstudio

EMStudio is a suite that is broken down into three programs, EMSTune, EMSLogView, and EMSLoad. The parent repository linked above is purely for the installer and releases, and should *never ever* be used to compile by *anyone*. I need to make a readme to make this more obvious. You should use one of the child projects listed below... or the installer executables in the EMStudio repository.

EMSTune
https://github.com/malcom2073/emstune
Basic tuning program. This is the main tuning application of EMStudio. This program is plugin based, allowing you to switch between plugins at runtime so you can connect to different ECUs without having to recompile/reinstall the main application. I have been working on getting the megasquirt protocol plugin up and running to communicate with Speeduino. It currently partially works, but it would certainly work with rusEFI as well.

EMSLogView
https://github.com/malcom2073/emslogview
Log viewing and graphing Able to load logs from LibreEMS and Megasquirt(At least, it could a few years ago), and display them:


EMSLoad (https://github.com/malcom2073/emsload) S19X firmware loading

It can load firmware to S19X based devices (LibreEMS). This is pretty useless for anything but S19X devices, extending support for other hardware would be fairly trivial, but at least in the case of speeduino it's pointless (arduino studio works for them, dunno how rusEFI does it)

EMStune Screenshots, since you can't actually do anything without hardware to connect to (No offline mode atm):
Image
Image
Image
Image
Image

EMSLogView screenshots:

Image
Image
Last edited by malcom2073 on Mon Dec 26, 2016 4:05 am, edited 1 time in total.
matt
donator
donator
Posts: 271
Joined: Fri Aug 26, 2016 11:32 am
Location: Malaysia

Re: EMStudio and rusEfi

Post by matt »

ahh.. got some clue right now. actually i am working at these 4 ecu (ms2, coolefi, speeduino, rusefi) and seems coolefi must use emstudio. will test it tomorrow.
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

@: Yep, coolefi uses EMStudio if you're using the LibreEMS firmware that afaik they all ship with. If you're on windows, there are binaries available here: https://github.com/malcom2073/emstudio/releases/download/daily_releases-12-g0f6e544/emstudio_windows.exe

I updated my post to clarify that the EMStudio repo is purely for releases. If you are on linux and wish to build EMStune yourself, check out from the EMSTune repository yourself, and use the unstable branch. Let me know what you're using/trying to do/etc, and I'll help as much as I can.
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: EMStudio and rusEfi

Post by AndreyB »

malcom2073 wrote:I have been working on getting the megasquirt protocol plugin up and running to communicate with Speeduino. It currently partially works, but it would certainly work with rusEFI as well.
How is Speeduino integration implemented - are you reading their .ini file or do you have some code defining the data structure format in some other way?

https://svn.code.sf.net/p/rusefi/code/trunk/firmware/tunerstudio/rusefi.ini is our file. How do I try it? :)
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
matt
donator
donator
Posts: 271
Joined: Fri Aug 26, 2016 11:32 am
Location: Malaysia

Re: EMStudio and rusEfi

Post by matt »

my platform is window 7 32bit. I am not familiar with linux. i tried link you're posted on libreeems and ecu got connected with pps show 49,50,51 repeatedly. just don't know how to configure it because i can't find where is basic setting, ve table, etc. or it need to press load from ecu to get those table?

fiy i have no idea about software, and i am more to tuner/ mechanics.
User avatar
kb1gtt
contributor
contributor
Posts: 3730
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: EMStudio and rusEfi

Post by kb1gtt »

Cool screen shots. It seems it's come along way sense I last saw it. Does it still split the front end and back end with TCP/IP pipe in the middle? I thought it was cool how you could remotely tune as long as I had the TCP/IP bit running on my end.

Some notes about rusEFI. Currently there is a script kind of program that generates the ms.ini file and some other file used by the rusEFI firmware. This script is handy as it keeps the ms.ini and firmware linked and prevents typo's and such hassles when you need to make changes. However at the end of the day, I think the EMStudio is interested in only the ms.ini file. I mention it because if rusEFI needs a change something in TS, it is very quick and easy to make the change. As well if there is some kind of configuration file for EMStudio, this script could probably generate that file.

What would we need to integrate EMStudio with rusEFI? Can it be done with the MS protocol, or should we look to get it communicating some other way?
Welcome to the friendlier side of internet crazy :)
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

russian wrote:How is Speeduino integration implemented - are you reading their .ini file or do you have some code defining the data structure format in some other way?
nt to
I am loading their ini file. I have not yet posted the megasquirt/speeduino code though, I can push it up to the repo if you're feeling adventurous and want to compile and try to run it, but all it does is read configuration at the moment and I'm not entirely sure how accurately it does that. I've not done more than a few hours of work on it :)
mattsabun wrote:i tried link you're posted on libreeems and ecu got connected with pps show 49,50,51 repeatedly. just don't know how to configure it because i can't find where is basic setting, ve table, etc. or it need to press load from ecu to
get those table?
One place I'm *really* lacking is end-user documentation on how to actually use EMStudio. When connecting, you should see the interrogation window:
Image

and then once connected, the Tunable Memory window should appear

Image

Which you can then use (by double clicking) to bring up tables for tuning. You should pop by the LibreEMS forums for support though, this isn't the place for it :)
kb1gtt wrote:Cool screen shots. It seems it's come along way sense I last saw it. Does it still split the front end and back end with TCP/IP pipe in the middle? I thought it was cool how you could remotely tune as long as I had the TCP/IP bit running on my end.
Nope, I don't recall how I had it working over TCP/IP, but the frontend/backend interfaces are over a c++ LoadLibrary plugin interface (using Qt's QPlugin).
kb1gtt wrote:What would we need to integrate EMStudio with rusEFI? Can it be done with the MS protocol, or should we look to get it communicating some other way?
It would work with the MS protocol initially, I'd just load the ms.ini file from rusEFI directly. I'd recommend working on an alternative communication protocol once MS integration is complete, as the MS protocol has some serious limitations unless you need to retain TS compatibility, there's no reason for using it.
matt
donator
donator
Posts: 271
Joined: Fri Aug 26, 2016 11:32 am
Location: Malaysia

Re: EMStudio and rusEfi

Post by matt »

i know i should post a questions there, and i have already done that. since you're also here and rusefi forum a bit active, so i should take this advantage. :D

this is interrogation runs well, i can see something is load between ecu and software
Image

but in this table, i see only first row is written afterstart enrichment. other row is written unknown until last row. i double click 1st row, it show something like hex i no idea what is it. how do i setup other row? it is automatic filled while interrogation or just simply type what features that i want?
Image

ps: Russian, kb1gtt: sorry for out of topic question. i hope this will useful for others to run rusefi with emstudio once it fully support.
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

I'll PM you then, to avoid cluttering this thread with this.
User avatar
kb1gtt
contributor
contributor
Posts: 3730
Joined: Tue Sep 10, 2013 1:42 am
Location: ME of USA

Re: EMStudio and rusEfi

Post by kb1gtt »

Not worries about cluttering this thread. Others here might learn something instead of asking the same questions in a couple days :)
Welcome to the friendlier side of internet crazy :)
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

kb1gtt wrote:Not worries about cluttering this thread. Others here might learn something instead of asking the same questions in a couple days :)
He doesn't need EMStudio support he needs LibreEMS support. The issue of Tunable memory indicating unknown and not having the VE/Lambda/Timing tables sounds like he's running an older firmware. In that case he needs to contact those guys and get his configuration ported to a more up to date firmware. I PMed him some log folders and the like that I'd like him to send me so I can verify firmware version.

FWIW, the MS plugin doesn't actually use the TunableMemory section, since that's not how TunerStudio does it. It uses the ParameterView to provide a menu-like interface similar to TunerStudio, which I'll grab some screenshots of in a bit.
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: EMStudio and rusEfi

Post by AndreyB »

Exciting to see new releases announced at https://www.facebook.com/EngineManagementStudio/ - just downloaded but LIbreEmsPlugin (twice) was the only choice :(
Attachments
screenshot.png
screenshot.png (63.02 KiB) Viewed 11714 times
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Indeed it is! I am working on MS protocol support as we speak!
Rhinoman
contributor
contributor
Posts: 256
Joined: Thu Sep 24, 2015 2:14 pm
Location: Wiltshire, UK

Re: EMStudio and rusEfi

Post by Rhinoman »

Is there any documentation on how to write a plug in?
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

There is not, I'm debating an architecture change at the moment... but if you're interested in Qt/QML/C++ development and looking to help, I'd be more than willing to have a chat.
Rhinoman
contributor
contributor
Posts: 256
Joined: Thu Sep 24, 2015 2:14 pm
Location: Wiltshire, UK

Re: EMStudio and rusEfi

Post by Rhinoman »

I've done some C++ development, mostly Borland but I've done a little MS Visual C++, its what I intend to use for future projects. I've dabbled with JSON for HTML and Android. I've used TunerPro for diagnostics and live tuning previously. I'd be interested in seeing how well EMStudio compares.
megatroneye
Posts: 18
Joined: Mon Jan 09, 2017 10:52 pm

Re: EMStudio and rusEfi

Post by megatroneye »

Hi,

@m thanks for your input so far.

I am just a candidate to become a contributor to RusEFI and I am thinking about giving a try to the RuseEFI-EMSTune integration, i.e making a draft implementation.

However, I don't know how much effort it would take.

Can you @ please share your knowledge about - what shall we do in order to make EMStune support RusEFI tuning? Some information that would let us estimate the effort in difficulty and time.
bri3d
Posts: 2
Joined: Fri Nov 26, 2021 11:45 pm
Github Username: bri3d
Slack: bri3d

Re: EMStudio and rusEfi

Post by bri3d »

Hi!

I was asked to post my thoughts about "live tuning" (i.e. integrated tune and write/log) capable frontend solutions here, under the framing of "well, we implemented ISO-TP and other standards, do we get any functionality for free because of it?"

I do have experience with live tuning, but unfortunately I bear bad news: the situation is not good and the current setup using the TunerStudio based system might be the best sadly.

I have been working on this for awhile for OEM ECUs (Simos18) with aftermarket tuning and it's a sad state of affairs. There's nothing I know of besides the TunerStudio protocol where you can implement a specified/defined protocol and get GUI functionality from a free or open source toolset "for free." Basically the issue is that while there are plenty of industry standards for flashing, logging, and calibration, they're only used by proprietary tools, either dealership or worse internal engineering only systems.

I'll brain dump what I know of beyond the GitHub projects already listed:

* UDS $22 readLocalIdentifier for logging. Factory dealership tools and clones for all ECUs throughout the industry use these parameters along with $23 readMemoryByAddress to log ECU values for diagnostics. Unfortunately... the mapping of Identifiers is proprietary per manufacturer and per ECU, so while this does gain "standards compliance," it doesn't gain anything in terms of immediate access to a good aftermarket logging, flashing, or editing tool, since each tool is vendor specific anyway. We wrote our own tool (SimosTools) for $22 logging on Simos18 because nothing else was any good.

* Implement CCP or XCP. These are the industry standards for calibration over CAN, and they define the exact kinds of primitives you would want for tuning. The problem is, it's a very proprietary and tight knit industry, so to calibrate on top of these requires a lot of proprietary tools that are illegal and hard to get. ASAP2Demo is the only free CCP/XCP calibration tool which I know of and while it's surprisingly good, it's not much better than TunerPro UI wise. Much like $22 and $23 UDS, this seems appealing on the surface because it's implementing an "industry standard," but at the end of the day it's an industry with thousands-of-dollars or fully proprietary tooling anyway, so there's really no point in compliance.

I guess if we were to implement our own GUI we could use CCP or XCP as the backing protocol, but there's really no value in complying with the thousands of pages of spec document in my mind?

* Implement a TunerPro RT DLL, and XDFs for a calibration layout. The backing transport for logging and live tuning / emulation can be whatever you want, you could write a TunerPro RT <-> TunerStudio DLL if you wanted, or invent your own transport protocol. He will give you the SDK if you ask, it's C++. This is fine but at the end of the day I don't find TunerPro RT to be much better than using TunerPro the "normal way" along with MegaLogViewer histograms. It's oriented around flat flash memory, but emulating a "calibration" layout wouldn't be hard.

* Implement a WinOLS OLS300 DLL. The backing transport can be whatever you want. You need good connections to get the SDK to do this, and OLS is not cheap. This is how high end commercial tuning tools like bFlash work. The WinOLS UI is very powerful, but outside of the European tuning scene, not many people will have it or want to afford it. Alternately: Implement RAM hacking against WinOLS. prj on nef made a tool that did this for awhile. Again, for European tuning scene folks, this would be _awesome_, but it leaves everyone else high and dry.

As an illustration of how bad the situation is, what we do to tune Simos18 is:

* We wrote our own RAM-based logging patch using a proprietary protocol on top of ISO-TP. There was no value to using the standards-based UDS protocols because none of the available logging applications offered any useful features beyond saving to a CSV.
* We wrote our own logger called SimosTools.
* We wrote a tool which automatically generates TunerPro XDF, ECUEdit XML, and WinOLS KP from vendor memory layout files (A2L).
* Depending on personal taste and desired spend, we use either TunerPro (good), ECUEdit (maybe better), or WinOLS (best) to edit the calibration binaries, and a tool I wrote to flash them.
* We use MegaLogViewer to tune based on logs - specifically, the histogram view is invaluable.
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

I've finally got the discretionary income available to purchase some hardware, so I've got a Nucleo STM32F7 board on order in order as of Friday to do some development. I'm also starting to look into what is required to run my vehicle on RusEFI on a Microrusefi or Proteus, so that'll be coming down the pipe in the next 6 months or so I hope.

I've been able to get EMS reading config information from the simulator using MS protocol and the INI file. I'm only able to display tables and curves, but thus far it's looking pretty promising that I can reuse most of the pieces of EMS with minimal effort to at least reproduce TS base level functionality of reading/writing tables, scalars, curves, and bits. The more complicated parts like diaogs and wizards will of course take significantly longer.

My plan is to convert EMS over to being RusEFI dedicated. This is partially because of a lack of interest in other projects using it, and partially because the effort of doing the plugin system and supporting the wide variety of memory access methodologies is a huge headache that greatly slowed down development on MS/RE support previously.

Of course this is all pending Andrey's thoughts about it... let me know if you've got other plans in the works and I'll stay out of the way and work on this quietly for my own systems :)
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: EMStudio and rusEfi

Post by AndreyB »

Yes, yes, yes please!

Is there an issue tracker on GH or anywhere to create a ticket for automatic .ini download from internet based on signature?
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

The github repo is still active ( https://github.com/malcom2073/emstudio ) I'll enable issue tickets on it and you can dump some ideas there
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: EMStudio and rusEfi

Post by AndreyB »

You have two at https://github.com/malcom2073/emstudio :)

thank you for considering rusEFI in your life.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Got Tables and (most) scalars tested and working solid. Byte ordering gave me a bit of a headache, but I believe it's all sorted out now!
image.png
image.png (155.8 KiB) Viewed 1091 times
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Got a whole lot more of the nuances of the ini file and panels/dialogs figured out. Also figured out how to do the bit fields properly, and I believe I'm reading proper values from the simulator across the board. 2d tables (Curves) still don't work right, and there's no actual WRITING of data, only reading at the moment, but I'm making significantly faster progress than I did last time I tired this for some reason. I'm super optimistic!
image.png
image.png (186.46 KiB) Viewed 1044 times
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Changed window layout to more match what people are familiar with, and added a serial/tcp connection dialog. Also started working on figuring out how to properly save data... this is going to require quite the number of unit tests to make sure I do it right for sure.
.
.
image.png
image.png (10.7 KiB) Viewed 990 times
image.png
image.png (218.86 KiB) Viewed 990 times
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

I've got writing capabilities for bits and scalars working. Testing with TS to verify that I'm writing correctly and I'm fairly confident I've got most of the math right!

I need to work up some unit tests for my bit/byte conversion functions, perhaps even writing up a mock ECU with a chunk of test memory so I can ensure changes don't break things.
User avatar
AndreyB
Site Admin
Posts: 13000
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: EMStudio and rusEfi

Post by AndreyB »

:geek:
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

https://github.com/malcom2073/emstudio/releases/tag/emstudio-unstable-20220702005247

Tables now read/write correctly!

Curves or live data gauges will be next, unsure which.
malcom2073
Posts: 22
Joined: Thu Aug 13, 2015 4:50 pm

Re: EMStudio and rusEfi

Post by malcom2073 »

Live data wound up being next. Mostly because I'm going to need to re-write the curve viewer from scratch... so this was an easier lift.
The dashboard is QML based, and generated at runtime, so it can be modified/rearranged/whatever without users needing to recompile
image.png
image.png (274.42 KiB) Viewed 864 times
Locked