Mount /var/logs as tmpfs, with help of overlayfs to save changes sometimes
Instead of just mounting tmpfs on /var/log I want to use overlayfs.
- /var/log are writable tmpfs, but containing files were there before
tmpfs mount. This old files are not in memory of tmpfs but in lower layer. - only changes are stored in tmpfs, while old and unmodified files
stored on SSD - sometimes it should be possible to write changes to SSD, for example
via cron. This should free up tmpfs memory
So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.
Point is to speed up a little, and safe SSD from many writes.
(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)
I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.
logs tmpfs overlayfs union-mount
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
add a comment |
Instead of just mounting tmpfs on /var/log I want to use overlayfs.
- /var/log are writable tmpfs, but containing files were there before
tmpfs mount. This old files are not in memory of tmpfs but in lower layer. - only changes are stored in tmpfs, while old and unmodified files
stored on SSD - sometimes it should be possible to write changes to SSD, for example
via cron. This should free up tmpfs memory
So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.
Point is to speed up a little, and safe SSD from many writes.
(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)
I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.
logs tmpfs overlayfs union-mount
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
add a comment |
Instead of just mounting tmpfs on /var/log I want to use overlayfs.
- /var/log are writable tmpfs, but containing files were there before
tmpfs mount. This old files are not in memory of tmpfs but in lower layer. - only changes are stored in tmpfs, while old and unmodified files
stored on SSD - sometimes it should be possible to write changes to SSD, for example
via cron. This should free up tmpfs memory
So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.
Point is to speed up a little, and safe SSD from many writes.
(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)
I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.
logs tmpfs overlayfs union-mount
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
Instead of just mounting tmpfs on /var/log I want to use overlayfs.
- /var/log are writable tmpfs, but containing files were there before
tmpfs mount. This old files are not in memory of tmpfs but in lower layer. - only changes are stored in tmpfs, while old and unmodified files
stored on SSD - sometimes it should be possible to write changes to SSD, for example
via cron. This should free up tmpfs memory
So, result should be: logs written to RAM, old and new boot logs accesable via same path. Changes are written sometimes to disk, by script.
Point is to speed up a little, and safe SSD from many writes.
(I saw similar thing in puppy linux, not for logs, but for all changes to root, but without installing it can't do the same, documentation not helps)
I will do same for browser cookies/cache based on answer. But persistent write will be done on browser close. Can't turn off browser cache, need at least small cache to have same bugs in my web development as users can have because of cache.
logs tmpfs overlayfs union-mount
logs tmpfs overlayfs union-mount
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
edited Nov 18 '18 at 9:31
Rui F Ribeiro
39.5k1479133
39.5k1479133
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
asked Feb 15 '18 at 10:46
LeonidMewLeonidMew
1066
1066
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.
#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs
#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay
#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log
To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.logrotateconfigs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
add a comment |
I can suggest anything-sync-daemon. It pretty good job.
https://wiki.archlinux.org/index.php/anything-sync-daemon
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f424341%2fmount-var-logs-as-tmpfs-with-help-of-overlayfs-to-save-changes-sometimes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.
#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs
#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay
#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log
To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.logrotateconfigs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
add a comment |
Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.
#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs
#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay
#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log
To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.logrotateconfigs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
add a comment |
Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.
#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs
#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay
#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log
To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
Managed to make /var/log overlay, it shows SSD log files, and changes. All changes are kept in RAM. Later I'll do syncing, so changes become permanent every hour, by copying upper layer to lower.
#prepare layers
sudo mkdir -p /var/log.tmpfs
sudo mount -t tmpfs -o rw,nosuid,nodev,noexec,relatime,size=512m,mode=0775 tmpfs /var/log.tmpfs
sudo mkdir -p /var/log.tmpfs/upper
sudo mkdir -p /var/log.tmpfs/work
sudo chown -R root:syslog /var/log.tmpfs
sudo chmod -R u=rwX,g=rwX,o=rX /var/log.tmpfs
#prepare overlay
sudo mkdir -p /var/log.overlay
sudo chown root:syslog /var/log.overlay
sudo chmod u=rwX,g=rwX,o=rX /var/log.overlay
#start overlay
sudo mount -t overlay -o rw,lowerdir=/var/log,upperdir=/var/log.tmpfs/upper,workdir=/var/log.tmpfs/work overlay /var/log.overlay
sudo mount --bind /var/log.overlay /var/log
To make changes persistent, its needed to unmount bind /var/log, copy files, then bind again.
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
answered Feb 16 '18 at 1:57
LeonidMewLeonidMew
1066
1066
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.logrotateconfigs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
add a comment |
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.logrotateconfigs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.
– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
why? what actual benefit do you gain from this? quickly fill up your RAM to trigger the kernel's OOM random-process-killer faster?
– cas
Feb 16 '18 at 4:54
Max size of tmpfs is set, so ram will not fill up.
logrotate configs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.– LeonidMew
Feb 16 '18 at 7:58
Max size of tmpfs is set, so ram will not fill up.
logrotate configs are tuned the way it never fill all reserved size. Anyway, I find out its not possible to make changes persistent online, so I have to deceide either remove this script or loose logs on reboot. This is normal for development server, not production.– LeonidMew
Feb 16 '18 at 7:58
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
I still don't get what you were trying to achieve. Where's the benefit? there's certainly no performance benefit. It just seems like extra complication and fragility for no good reason. Probably qualifies as some form of premature optimisation: maybe something like "ram disks are fast so i'll put my logs on a tmpfs" without testing whether there's any actual, noticeable benefit. Hint: unless the logs are fsynced on every write, writes will be buffered anyway.
– cas
Feb 16 '18 at 8:02
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
SSD have large buffer it self, I just don't want that unnecessary writes. Ok, I get your point. Will not recommend it somebody, but stay with it for myself :)
– LeonidMew
Feb 16 '18 at 8:10
add a comment |
I can suggest anything-sync-daemon. It pretty good job.
https://wiki.archlinux.org/index.php/anything-sync-daemon
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I can suggest anything-sync-daemon. It pretty good job.
https://wiki.archlinux.org/index.php/anything-sync-daemon
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
I can suggest anything-sync-daemon. It pretty good job.
https://wiki.archlinux.org/index.php/anything-sync-daemon
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I can suggest anything-sync-daemon. It pretty good job.
https://wiki.archlinux.org/index.php/anything-sync-daemon
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 13 mins ago
GinnunGinnun
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
answered 13 mins ago
GinnunGinnun
1
answered 13 mins ago
GinnunGinnun
1
1
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Ginnun is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f424341%2fmount-var-logs-as-tmpfs-with-help-of-overlayfs-to-save-changes-sometimes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
