"Variable penalty should not be negative!" with in-flight messages and bandwidth profiles
I have issues using bandwidth profiles, I keep getting a "Variable penalty should not be negative!" error, which causes the simulation to abort. I can actually trigger the same issue with latency profiles.
What I'm doing is:
- I have 1 Load producer
- I have 8 workers, split into 2 NetZones that each have their own mailbox
- The load producer sends 1000 bytes messages every second in a round-robin fashion to each of the 8 workers using
Mailbox::put_async
- The workers read the messages with
Mailbox::get
- After 1 second, I reduce the bandwidth to NetZone A to something really small (200 bytes per second). This basically causes a monster traffic jam on that link, where all the workers in NetZone A are competing to read the messages from a saturated link. There is always at least one message in flight on that link.
- After 100 seconds (before the traffic jam is resolved) I return that link to its regular bandwidth (42MBps)
- Then I get the
Variable penalty should not be negative!
error, and the simulation is aborted.
I suspect this is caused by the inflight (or maybe the queued) messages suddenly finding that they have a lot more bandwidth available, and struggle to calculate their new arrival time.
Here's a toy project that displays the problem. It's my first C++ project in 20 years, so everything might not be up to standard, but it's enough to display the problem for me.