Realtime Scheduling and hanging processes
We run some software on various systems at realtime priority (SCHED_FIFO, Priority: 99) To hit certain frame requirements in the software our engineers work on.
one of our side projects doesn't "really" need the same realtime performance, but they need some functionality that is only achievable when running in realtime. IE frame misses aren't a critical failure state, but they need to try and hit in order for certain other things to happen properly. It's a confusing mess.
Anyway, While running the software realtime, they also use the system for other quick tasks, often opening terminal windows, running a quick program, then Ctrl+d to drop the terminal, and move on with their life. Doing this leaves copies of the applications in an un-interrupt able sleep, that over time eats up system resources and locks up the system. If the software is stopped, it immediately resolves all of those sleeping processes and finishes closing everything.
My thoughts are that whatever cleanup process runs to finish cleaning up those processes simply wont complete with the system holding realtime priority over things. I realize "this isn't how you should use a realtime system" and all of that, but unfortunately its an issue that they refuse to concede the point on, and I have to find a magical solution to a simple problem(don't use a realtime system as a workstation.)
So my question is: Is there a way of manually forcing the cleanup process from time to time to take precedence over the maximum running process?
I've considered trying to get fancy with cgroups but that seems like a really complicated solution to the problem, and one that I'm not sure will actually work even if i take the time to learn how to work well with cgroups and the various kernel timing stats
rhel real-time
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
add a comment |
We run some software on various systems at realtime priority (SCHED_FIFO, Priority: 99) To hit certain frame requirements in the software our engineers work on.
one of our side projects doesn't "really" need the same realtime performance, but they need some functionality that is only achievable when running in realtime. IE frame misses aren't a critical failure state, but they need to try and hit in order for certain other things to happen properly. It's a confusing mess.
Anyway, While running the software realtime, they also use the system for other quick tasks, often opening terminal windows, running a quick program, then Ctrl+d to drop the terminal, and move on with their life. Doing this leaves copies of the applications in an un-interrupt able sleep, that over time eats up system resources and locks up the system. If the software is stopped, it immediately resolves all of those sleeping processes and finishes closing everything.
My thoughts are that whatever cleanup process runs to finish cleaning up those processes simply wont complete with the system holding realtime priority over things. I realize "this isn't how you should use a realtime system" and all of that, but unfortunately its an issue that they refuse to concede the point on, and I have to find a magical solution to a simple problem(don't use a realtime system as a workstation.)
So my question is: Is there a way of manually forcing the cleanup process from time to time to take precedence over the maximum running process?
I've considered trying to get fancy with cgroups but that seems like a really complicated solution to the problem, and one that I'm not sure will actually work even if i take the time to learn how to work well with cgroups and the various kernel timing stats
rhel real-time
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
If you are referring to zombie processes, you need to callsigaction()to ignoreSIGCHLDin your application.
– meuh
Apr 22 '16 at 6:47
add a comment |
We run some software on various systems at realtime priority (SCHED_FIFO, Priority: 99) To hit certain frame requirements in the software our engineers work on.
one of our side projects doesn't "really" need the same realtime performance, but they need some functionality that is only achievable when running in realtime. IE frame misses aren't a critical failure state, but they need to try and hit in order for certain other things to happen properly. It's a confusing mess.
Anyway, While running the software realtime, they also use the system for other quick tasks, often opening terminal windows, running a quick program, then Ctrl+d to drop the terminal, and move on with their life. Doing this leaves copies of the applications in an un-interrupt able sleep, that over time eats up system resources and locks up the system. If the software is stopped, it immediately resolves all of those sleeping processes and finishes closing everything.
My thoughts are that whatever cleanup process runs to finish cleaning up those processes simply wont complete with the system holding realtime priority over things. I realize "this isn't how you should use a realtime system" and all of that, but unfortunately its an issue that they refuse to concede the point on, and I have to find a magical solution to a simple problem(don't use a realtime system as a workstation.)
So my question is: Is there a way of manually forcing the cleanup process from time to time to take precedence over the maximum running process?
I've considered trying to get fancy with cgroups but that seems like a really complicated solution to the problem, and one that I'm not sure will actually work even if i take the time to learn how to work well with cgroups and the various kernel timing stats
rhel real-time
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
We run some software on various systems at realtime priority (SCHED_FIFO, Priority: 99) To hit certain frame requirements in the software our engineers work on.
one of our side projects doesn't "really" need the same realtime performance, but they need some functionality that is only achievable when running in realtime. IE frame misses aren't a critical failure state, but they need to try and hit in order for certain other things to happen properly. It's a confusing mess.
Anyway, While running the software realtime, they also use the system for other quick tasks, often opening terminal windows, running a quick program, then Ctrl+d to drop the terminal, and move on with their life. Doing this leaves copies of the applications in an un-interrupt able sleep, that over time eats up system resources and locks up the system. If the software is stopped, it immediately resolves all of those sleeping processes and finishes closing everything.
My thoughts are that whatever cleanup process runs to finish cleaning up those processes simply wont complete with the system holding realtime priority over things. I realize "this isn't how you should use a realtime system" and all of that, but unfortunately its an issue that they refuse to concede the point on, and I have to find a magical solution to a simple problem(don't use a realtime system as a workstation.)
So my question is: Is there a way of manually forcing the cleanup process from time to time to take precedence over the maximum running process?
I've considered trying to get fancy with cgroups but that seems like a really complicated solution to the problem, and one that I'm not sure will actually work even if i take the time to learn how to work well with cgroups and the various kernel timing stats
rhel real-time
rhel real-time
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
edited 1 hour ago
Rui F Ribeiro
39.6k1479132
39.6k1479132
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
asked Apr 21 '16 at 21:40
GravyGravy
1,363521
1,363521
If you are referring to zombie processes, you need to callsigaction()to ignoreSIGCHLDin your application.
– meuh
Apr 22 '16 at 6:47
add a comment |
If you are referring to zombie processes, you need to callsigaction()to ignoreSIGCHLDin your application.
– meuh
Apr 22 '16 at 6:47
If you are referring to zombie processes, you need to call
sigaction() to ignore SIGCHLD in your application.– meuh
Apr 22 '16 at 6:47
If you are referring to zombie processes, you need to call
sigaction() to ignore SIGCHLD in your application.– meuh
Apr 22 '16 at 6:47
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f278223%2frealtime-scheduling-and-hanging-processes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f278223%2frealtime-scheduling-and-hanging-processes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
If you are referring to zombie processes, you need to call
sigaction()to ignoreSIGCHLDin your application.– meuh
Apr 22 '16 at 6:47