Plugin link_energy fails with timeout communications
Hello!
I am currently working with simgrid and the plugins host_energy
and link_energy
and I may have found a bug concerning the last one. At some point in my program, 2 hosts can end up trying to send a message to each other resulting to a deadlock. To fix this problem I used the timeout argument of the method Mailbox::put()
on one of the Host. This solution works great if the link_energy
plugin is disabled, but when it is enabled it crashes (note that the same problem happens when using put_async
with wait_for
).
Here is a small example to demonstrate the bug:
static void worker(std::vector<std::string> args)
{
auto master_mailbox = simgrid::s4u::Mailbox::by_name("master_mailbox");
auto worker_mailbox = simgrid::s4u::Mailbox::by_name("worker_mailbox");
std::string payload = std::string("Hello from worker");
XBT_INFO("Worker sending to master");
try
{
master_mailbox->put(&payload, 2, 10); // try to sent with timeout
}
catch (simgrid::TimeoutException) {
XBT_INFO("TimeoutException");
}
std::string *res = worker_mailbox->get<std::string>();
XBT_INFO("%s", res->c_str());
}
static void master(std::vector<std::string> args)
{
auto master_mailbox = simgrid::s4u::Mailbox::by_name("master_mailbox");
auto worker_mailbox = simgrid::s4u::Mailbox::by_name("worker_mailbox");
std::string *payload = new std::string("Hello from master");
XBT_INFO("Master sending to worker");
worker_mailbox->put(payload, 2); // Send without timeout -> master have the priority
}
int main(int argc, char* argv[])
{
simgrid::s4u::Engine e(&argc, argv);
sg_link_energy_plugin_init();
/* Register the functions representing the actors */
e.register_function("master", &master);
e.register_function("worker", &worker);
/* Load the platform description and then deploy the application */
e.load_platform(argv[1]);
e.load_deployment(argv[2]);
/* Run the simulation */
e.run();
XBT_INFO("Simulation is over");
return 0;
}
With link_energy disabled:
[Tremblay:master:(1) 0.000000] [s4u_app_masterworker/INFO] Master sending to worker
[Jupiter:worker:(2) 0.000000] [s4u_app_masterworker/INFO] Worker sending to master
[Jupiter:worker:(2) 10.000000] [s4u_app_masterworker/INFO] TimeoutException
[Jupiter:worker:(2) 10.019015] [s4u_app_masterworker/INFO] Hello from master
[10.019015] [s4u_app_masterworker/INFO] Simulation is over
With link_energy enabled:
[Tremblay:master:(1) 0.000000] [s4u_app_masterworker/INFO] Master sending to worker
[Jupiter:worker:(2) 0.000000] [s4u_app_masterworker/INFO] Worker sending to master
[10.000000] ./src/kernel/activity/CommImpl.cpp:171: [root/CRITICAL] You cannot use get_traversed_links() if your communication is not ready (WAITING)
Backtrace (displayed in actor maestro):
-> #0 xbt_backtrace_display_current in /usr/lib/libsimgrid.so.3.35
-> #1 simgrid::kernel::activity::CommImpl::get_traversed_links() const in /usr/lib/libsimgrid.so.3.35
-> #2 0x00007D7211AF405B in /usr/lib/libsimgrid.so.3.35
-> #3 simgrid::kernel::activity::CommImpl::finish() in /usr/lib/libsimgrid.so.3.35
-> #4 simgrid::kernel::EngineImpl::handle_ended_actions() const in /usr/lib/libsimgrid.so.3.35
-> #5 simgrid::kernel::EngineImpl::run(double) in /usr/lib/libsimgrid.so.3.35
-> #6 main in ./test-plugin-link-bug
After investigating into the plugin's source code it seems like the function on_communication
, line 170, causes the error because it calls get_traversed_links
which raises this error: xbt_assert(get_state() != State::WAITING, "You cannot use %s() if your communication is not ready (%s)", __func__, get_state_str());
As a quick fix I added a if testing for communication state on the callback function:
static void on_communication(const simgrid::s4u::Comm& comm)
{
// Nasty fix because I cannot manage to import Activity::State
if ((int) comm.get_state() == 2)
return;
const auto* pimpl = static_cast<simgrid::kernel::activity::CommImpl*>(comm.get_impl());
for (auto const* link : pimpl->get_traversed_links()) {
if (link != nullptr && link->get_sharing_policy() != simgrid::s4u::Link::SharingPolicy::WIFI) {
XBT_DEBUG("Update %s on Comm Start/End", link->get_cname());
link->extension<LinkEnergy>()->update();
}
}
}
Note that I got the value 2 by using an async put (which have exactly the same bug) and printing the value of my communication object when the timeout exception was caught. So I suppose 2 equals to waiting.
Furthermore the function on_communication
is added as a callback for Comm::on_start_cb
and Comm::on_completion_cb
.
By duplicating the function, assigning the copy to on_completion
and adding/removing the fix I was able to identify that only on_completion
callback is responsible for the error. I don't know a lot about simgrid's source code but I guess that makes sense considering the bug only occurs on timeout communications.
Am I wrong in the first place by fixing my "two way sending conflict" problem using timeouts?
If not can we fix this problem in a better way?
Should we take into account the energy of a timeout'd communication? Because right now I just ignore the problem...
Thanks in advance for your help.