Unit tests

It's all about the code!
Post Reply
megatroneye
Posts: 18
Joined: Mon Jan 09, 2017 10:52 pm

Unit tests

Post by megatroneye »

Hi,

what about using a framework for unit tests? They offer out-of-the-box functionality for several repetitive tasks (e.g. assertions), annotations (useful for integration with other tools, such as code coverage or continuous integration servers). Moreover - they display tests results in user-friendly ways. In fact, they render reports, not only for errors but also for successes.

For example, I can recommend CppUTest. I've been using it for several months in commercial projects. It's easy to use. When you execute your tests, they can be displayed on Eclipse like this:

https://raw.githubusercontent.com/arstrube/images/master/cdt.testrunner.cpputest/eclipse-cpputest-3-ran-3.png

And CppUTest can just be added on top of the existing tests, I think. By annotating them. At least I think so.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

What problem are you trying to solve?

We have https://github.com/rusefi/rusefi/tree/master/unit_tests without any external framework, and jenkins to run it, with results published in github https://github.com/rusefi/rusefi/commits/master
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
megatroneye
Posts: 18
Joined: Mon Jan 09, 2017 10:52 pm

Re: Unit tests

Post by megatroneye »

russian wrote:What problem are you trying to solve?

We have https://github.com/rusefi/rusefi/tree/master/unit_tests without any external framework, and jenkins to run it, with results published in github https://github.com/rusefi/rusefi/commits/master
The ones implied on the advantages of test frameworks I mentioned - the manipulation of tests in a computable way, for example for seamless integration (not manual integration) with Jenkins.

Can you please paste here an example of a test report? Thanks in advance!
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

With very little human resources available I am not convinced that switching from current bare-bones unit test framework to a better one is justified - at the moment I am not aware of any specific advantages worth the trouble.

There is no report at the moment - at the moment there is only process exit code, it's either 0 for sucess or not zero for failure.
current gap 11.666600/0.090909/33.000000 c=174999 prev=15000

TT_NISSAN_SR20VE_360 isSynchronizationPoint=1 index=183 SHAFT_PRIMARY_RISING

nextTriggerEvent index=184

index=0 3

All triggers exported to triggers.txt


C:\Program Files (x86)\Jenkins\workspace\rusEfi unit tests from GITHUB\unit_tests\build>IF NOT ERRORLEVEL echo UNIT TEST FAILED
echo was unexpected at this time.

C:\Program Files (x86)\Jenkins\workspace\rusEfi unit tests from GITHUB\unit_tests\build>IF NOT ERRORLEVEL 0 EXIT /B 1

C:\Program Files (x86)\Jenkins\workspace\rusEfi unit tests from GITHUB\unit_tests\build>cd ..
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

Things have changes since 2017, we now use https://github.com/google/googletest framework

By the way test_idle_controller.cpp would be a typical simple unit test.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
User avatar
md_sanci
Posts: 69
Joined: Thu May 30, 2019 6:13 am
Location: Romania
Github Username: shadowm60
Slack: Alex M
Contact:

Re: Unit tests

Post by md_sanci »

Hello all,

For building the SW there is somehow a description but what i managed to do is use eclipse for code and build it with eclipse, here have to take care of passing the arguments correctly, i learned that .bat files are ok and describing how to do it.

For unit tests, i copied them (actually everything) on workspace folder and tried to do unti tests.

there is a jenkins file, but i cannot seem to execute it correctly, i have installed the compiler it needs, but still having some issues.

Is there a way to make execute them? i would like to execute them all, then add my code and see if the report is ok and add some extra stuff? Or should i just spend more time reading about the google test framework?
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

cd unit_tests
make
build/rusefi_test.exe

and that would run unit tests!

You do not need anything related to Jenkins to run tests locally and manually.

as for adding new tests, just add new method like "TEST(util, crc) {"
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
rubwen
Posts: 4
Joined: Fri Jun 05, 2020 9:03 am

Re: Unit tests

Post by rubwen »

Hi,
I've spent some time to figure out how to compile the rusEFI-Code with the right version of Make and the GNU Arm Embedded Toolchain, this works fine now (despite the 17 warnings), but I am not able to compile the unit_tests. I didn't make any changes to it, I've got the repository-files from April 12th 2022. Besides the displayed version of Java SDK I've also tried versions JDK 1.7 and JDK 1.8, which lead to an "unexpected '('" error. I am using Windows 10. Can you tell me what I'm doing wrong? Are there more details I can provide? Thank you in advance!
Attachments
make-version, java-version and compiling error
make-version, java-version and compiling error
forum_make_unit_tests_error.jpg (411.38 KiB) Viewed 4441 times
mck1117
running engine in first post
running engine in first post
Posts: 1493
Joined: Mon Jan 30, 2017 2:05 am
Location: Seattle-ish

Re: Unit tests

Post by mck1117 »

rubwen wrote:
Tue May 24, 2022 4:30 pm
Hi,
I've spent some time to figure out how to compile the rusEFI-Code with the right version of Make and the GNU Arm Embedded Toolchain, this works fine now (despite the 17 warnings), but I am not able to compile the unit_tests. I didn't make any changes to it, I've got the repository-files from April 12th 2022. Besides the displayed version of Java SDK I've also tried versions JDK 1.7 and JDK 1.8, which lead to an "unexpected '('" error. I am using Windows 10. Can you tell me what I'm doing wrong? Are there more details I can provide? Thank you in advance!
I think there's no guarantee that it will compile at all on Windows at this point. It's much easier to get it set up if you use WSL: https://docs.microsoft.com/en-us/windows/wsl/about
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

"Files" seems to be part of "Program Files" with a space which seems to not work well

progra~1 would often work instead of "program files" as a folder name without a space.

Matt, it would _totally_ compile on Windows for sure.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
mck1117
running engine in first post
running engine in first post
Posts: 1493
Joined: Mon Jan 30, 2017 2:05 am
Location: Seattle-ish

Re: Unit tests

Post by mck1117 »

AndreyB wrote:
Tue May 24, 2022 6:35 pm
Matt, it would _totally_ compile on Windows for sure.
I'm sure it would, but it's going to be easier and more performant on WSL or "real" Linux.
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
rubwen
Posts: 4
Joined: Fri Jun 05, 2020 9:03 am

Re: Unit tests

Post by rubwen »

AndreyB wrote:
Tue May 24, 2022 6:35 pm
"Files" seems to be part of "Program Files" with a space which seems to not work well
This was it, thank you very much, I should have seen that myself. Is there some kind of documentation about the unit tests? I didn't find any information about it besides this thread. I compiled the unit_tests, ran the rusefi_test.exe and got the trigger images from the triggers.txt, but I am not sure what to do with these results. The last part of rusefi_test.exe tells me which tests have failed, but without a hint of what went wrong.


FYI, the rusefi_console.jar path in the last line of firmware\gen_trigger_images.bat didn't work for me because there isn't a java_console_binary subfolder, the rusefi_console.jar was located somewhere else:
java -cp ../java_console_binary/rusefi_console.jar com.rusefi.TriggerImage
Attachments
Result of rusefi_test.exe
Result of rusefi_test.exe
forum_rusefi_test_errors.jpg (64.28 KiB) Viewed 4296 times
User avatar
AndreyB
Site Admin
Posts: 14292
Joined: Wed Aug 28, 2013 1:28 am
Location: Jersey City
Github Username: rusefillc
Slack: Andrey B

Re: Unit tests

Post by AndreyB »

rubwen wrote:
Wed Jun 01, 2022 2:49 pm
The last part of rusefi_test.exe tells me which tests have failed, but without a hint of what went wrong.
I read this as "I choose to ignore all the information on top and now I have no hints at what is wrong".

Please do not ignore all the infomation on top?

I see that you run unit tests without an IDE and that's strange.
Very limited telepathic abilities - please post logs & tunes where appropriate - http://rusefi.com/s/questions

Always looking for C/C++/Java/PHP developers! Please help us see https://rusefi.com/s/howtocontribute
mck1117
running engine in first post
running engine in first post
Posts: 1493
Joined: Mon Jan 30, 2017 2:05 am
Location: Seattle-ish

Re: Unit tests

Post by mck1117 »

AndreyB wrote:
Wed Jun 01, 2022 3:03 pm
I see that you run unit tests without an IDE and that's strange.
not strange, there are dozens of us!
rubwen wrote:
Wed Jun 01, 2022 2:49 pm
Is there some kind of documentation about the unit tests? I didn't find any information about it besides this thread. I compiled the unit_tests, ran the rusefi_test.exe and got the trigger images from the triggers.txt, but I am not sure what to do with these results. The last part of rusefi_test.exe tells me which tests have failed, but without a hint of what went wrong.
we use https://en.wikipedia.org/wiki/Google_Test, maybe start there
Post Reply