Current performance of ignition robotic libraries #20
Comments
|
Original comment by Diego Ferigo (Bitbucket: dgferigo).
|
|
Original comment by Diego Ferigo (Bitbucket: dgferigo).
Updated the cachegrind file with a longer acquisition |
|
Original comment by Michael Carroll (Bitbucket: Michael Carroll, GitHub: mjcarroll). Thanks for reporting. We're currently in the process of doing some more detailed analysis of performance and eliminating bottlenecks, so this is useful information. Just a few questions:
One thing you can try since you are not using the GUI, you can disable the SceneBroadcaster system plugin, since that is only used for the GUI updates. |
|
Original comment by Diego Ferigo (Bitbucket: dgferigo). Hi Michael, thanks for the prompt response! The project is open source, you can find it here: https://github.com/robotology/gym-ignition.
I didn't add / remove any sleep wrt upstream. Did you mean anything in particular?
CartPole models,
I think I disabled it here
Thank for the tip. I can actually enable / disable the GUI, and I thought that it only introduces overhead to report this issue. I can provide you another cachegrind analysis without it if needed. |
|
Original comment by Michael Carroll (Bitbucket: Michael Carroll, GitHub: mjcarroll). Also, based on your longer log, it looks like you may be calling |
|
Original comment by Diego Ferigo (Bitbucket: dgferigo). You are right, I call it many times in here (function body I'll try to optimize it e.g. by setting the force to 0 in the |
|
Original comment by Diego Ferigo (Bitbucket: dgferigo). @mjcarroll As suggested I disabled all the GUI-related systems (broadcaster and user-commands) and commented out the GUI plugins (scene 3d, world control and world stats). The two cases represented by removing the component and zeroing its value did not differ significantly. The output of
I did not do any statistical significant analysis, more or less every run has similar values. However, this time the callgrind analysis is slightly better. I attached the 5% and 2% graphs. Now the |
|
Original comment by Diego Ferigo (Bitbucket: dgferigo).
Added callgrind files after disabling the GUI plugins and preventing the component to be removed every step. |
|
Original comment by Diego Ferigo (Bitbucket: dgferigo).
Cropped the images |
|
I'd like to have a completion criterion for this issue. Performance is quite broad: in theory, things can always go faster, and Gazebo has lots of features that we could focus on. The conversation seems to be focused on physics, so I propose reducing the scope of this issue to answering: Is the overhead added by Ignition on top of a physics engine unacceptable? This question is hard enough to answer with minimal functionality, so I propose the following assumptions to get to test the core functionality, without optional features:
Here's what
Here's what it looks like with TPE, which is much faster:
|
|
Hey there, recently I came across this cppcon talk Performance Matters by Emery Berger who talks about performance evaluation and something he calls "causal profiling". Since the 2021 Q2 focus is on performance and this issue (at least from the title) is about performance of the robotic libraries, I wanted to ask what you think about profiling Gazebo with coz to identify improvements possibilities. I also wanted to ask how you plan to evaluate the changes of this Q2 roadmap item and if it might be worth considering Stabilizer to do so. Cheers, |
Thank you for the tips, @stefanbuettner. A causal profiler looks really interesting to help us identify where to best focus our efforts. We've been using Ignition Common's profiler wrapper around Remotery, that's what I used on the comment above. Our profiler API is meant to be abstract enough so we can plug other profilers, but it looks to me that the nature of a causal profiler is different enough that we shouldn't just wrap
We've been using the Remotery profiler as well as real time factor to evaluate changes as we make them. Stabilizer seems to offer a more rigorous approach that could be interesting. |
|
We've been working on setting up Context
So, before embarking into an implementation that is able to live in tandem with the Setting up cozAfter struggling a bit with the installation of
Note: In case it is helpful we tried this in a container using bionic.
cd colcon_ws/src
git clone [email protected]:plasma-umass/coz.gitNote:
sudo apt-get install build-essential cmake docutils-common libelfin-dev nodejs npm python3
colcon build --packages-select coz
find_package(coz-profiler REQUIRED HINTS "${CMAKE_INSTALL_PREFIX}/lib/cmake")Note1: even though Note2: A hint must be passed to find the
So tipically: set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -gdwarf-3")Note: There isn't clear information about this issue in the
target_include_directories(${PROJECT_LIBRARY_TARGET_NAME}
PUBLIC
...
$<BUILD_INTERFACE:${CMAKE_INSTALL_PREFIX}/include>
)
#remember to source install to also get the coz command
source install/setup.bash
coz run --- ./<path_to_your_executable/program/simulation>
#use sudo and the entire path to coz if needed.eg.:
sudo ./install/bin/coz run --- ./<path_to_your_executable/program/simulation>a Note: As you may noticed it, we haven't linked against Current/next steps:
|
|
Great work, @francocipollone ! It's unfortunate that you had to jump through so many hoops to get started using |
|
Some updates on this front Setting Progress pointscoz profiler lets you select one or several "progress points", these are points in the code that you would like to improve and when you run the profiler, coz inspects the code and provides information about which line or entity you have to improve in order to get better performance at the "progress points" before selected. To start, we set a
Click to see ign-gazebo/src/CMakeLists.txttarget_link_libraries(${PROJECT_LIBRARY_TARGET_NAME}
PUBLIC
ignition-math${IGN_MATH_VER}
ignition-plugin${IGN_PLUGIN_VER}::core
ignition-common${IGN_COMMON_VER}::ignition-common${IGN_COMMON_VER}
ignition-common${IGN_COMMON_VER}::profiler
ignition-fuel_tools${IGN_FUEL_TOOLS_VER}::ignition-fuel_tools${IGN_FUEL_TOOLS_VER}
ignition-gui${IGN_GUI_VER}::ignition-gui${IGN_GUI_VER}
ignition-transport${IGN_TRANSPORT_VER}::ignition-transport${IGN_TRANSPORT_VER}
sdformat${SDF_VER}::sdformat${SDF_VER}
protobuf::libprotobuf
coz::coz
dl
PRIVATE
ignition-plugin${IGN_PLUGIN_VER}::loader
)BuildingAs Running cozWe created an executable that in essence set up a Then we ran coz doing: While running, information is being dumped to a When we plot the dumped information no interesting results are given because no information about ignition files are provided.
During this run, Click to see coz loggingEffectively, there are no ignition-related files being inspected (but some ignition math files ) Trying an alternative.I've checked the So tried another thing. Instead of creating an executable that links to However, this run ended up aborted so the Some comments
I will try to dig a bit deeper into this to see if I can get a real profile. |
Original report (archived issue) by Diego Ferigo (Bitbucket: dgferigo).
The original report had attachments: cachegrind.png, callgrind_2.png, callgrind_5.png
Summary
The overhead introduced by ignition gazebo seems significant. The time spent to compute physics is a small fraction of the total time.
Description
First off, thanks for all the work you're doing! Despite their early development status, ignition libraries are amazing.
I am currently developing an application based on ignition robotics. Starting by saying that it is still at a very early stage of development, I tried to evaluate its performance in order to catch initial bottlenecks. The test I did is a simple cart pole simulation with ignition configured in such a way that it tries to go as fast as it can. Currently, there's only one thread, and I can see that the CPU goes to 100% when the application is executing.
Referring to the attached cachegrind image, I am surprised to see that only the ~11% of the time passed inside the ignition libraries (the entry point is Server::Run) is actually spent in the physics engine (currently, dart). This means that the remaining 89% is computation overhead.
Is this behavior normal to your eyes? I saw that you performed in the past some profiling, maybe you have some insights on it.
Note: this is a headless simulation, I do not use the
ign-gazebo-guiprocessThe text was updated successfully, but these errors were encountered: