[ptxdist] Race condition between rc-once.service and systemd-remount-fs.service

[ptxdist] Race condition between rc-once.service and systemd-remount-fs.service

[Ian Abbott]

Hi,

If the root file system is initially mounted read-only, there is a 
possible race between rc-once.service and the 
systemd-remount-fs.service.  If rc-once.service sees the root filesystem 
is mounted read-only, then it will temporarily remount it read-write and 
later mount it read-only again.  Meanwhile, systemd-remount-fs.service 
will also remount the root file system as specified by /etc/fstab which 
could be read-write. Depending on the ordering, it is possible for the 
root file-system to end up mounted read-only when it should have ended 
up mounted read-write.

As long as the two services do not run simultaneously, the root file 
system should end up mounted in the correct state.  However, I do not 
know what the intended order is supposed to be.  Perhaps it is 
intentionally system-dependent?

-- 
-=( Ian Abbott <abbotti@mev.co.uk> || MEV Ltd. is a company  )=-
-=( registered in England & Wales.  Regd. number: 02862268.  )=-
-=( Regd. addr.: S11 & 12 Building 67, Europa Business Park, )=-
-=( Bird Hall Lane, STOCKPORT, SK3 0XA, UK. || www.mev.co.uk )=-

Re: [ptxdist] Race condition between rc-once.service and systemd-remount-fs.service

[Michael Olbrich]

On Fri, May 03, 2024 at 07:33:32PM +0100, Ian Abbott wrote:
> If the root file system is initially mounted read-only, there is a possible
> race between rc-once.service and the systemd-remount-fs.service.  If
> rc-once.service sees the root filesystem is mounted read-only, then it will
> temporarily remount it read-write and later mount it read-only again.
> Meanwhile, systemd-remount-fs.service will also remount the root file system
> as specified by /etc/fstab which could be read-write. Depending on the
> ordering, it is possible for the root file-system to end up mounted
> read-only when it should have ended up mounted read-write.
> 
> As long as the two services do not run simultaneously, the root file system
> should end up mounted in the correct state.  However, I do not know what the
> intended order is supposed to be.  Perhaps it is intentionally
> system-dependent?

It's not intentional. I just never noticed because I always keep the rootfs
read-only by default. I think rc-once.service should have
'After=systemd-remount-fs.service'. That should fix the problem, right?

Michael

-- 
Pengutronix e.K.                           |                             |
Steuerwalder Str. 21                       | http://www.pengutronix.de/  |
31137 Hildesheim, Germany                  | Phone: +49-5121-206917-0    |
Amtsgericht Hildesheim, HRA 2686           | Fax:   +49-5121-206917-5555 |

Re: [ptxdist] Race condition between rc-once.service and systemd-remount-fs.service

[Ian Abbott]

On 09/05/2024 10:34, Michael Olbrich wrote:
> On Fri, May 03, 2024 at 07:33:32PM +0100, Ian Abbott wrote:
>> If the root file system is initially mounted read-only, there is a possible
>> race between rc-once.service and the systemd-remount-fs.service.  If
>> rc-once.service sees the root filesystem is mounted read-only, then it will
>> temporarily remount it read-write and later mount it read-only again.
>> Meanwhile, systemd-remount-fs.service will also remount the root file system
>> as specified by /etc/fstab which could be read-write. Depending on the
>> ordering, it is possible for the root file-system to end up mounted
>> read-only when it should have ended up mounted read-write.
>>
>> As long as the two services do not run simultaneously, the root file system
>> should end up mounted in the correct state.  However, I do not know what the
>> intended order is supposed to be.  Perhaps it is intentionally
>> system-dependent?
> 
> It's not intentional. I just never noticed because I always keep the rootfs
> read-only by default. I think rc-once.service should have
> 'After=systemd-remount-fs.service'. That should fix the problem, right?

That's what I have done locally and it seems to work.

One slightly related tip for people using systemd is to avoid using 
busybox's 'mount' command when systemd is being used. Use the mount 
command from util-linux instead).  This is because busybox's mount 
command does not read the /etc/fstab file for remount operations, it 
only reads the mtab file.  For example, if the root file system is 
currently mounted read-only, but is not marked as read-only in 
/etc/fstab, the systemd-remount-fs.service will run `mount / -o 
remount`, and busybox's mount will remount it read-only again, whereas 
util-linux's mount will remount it read-write as intended.

-- 
-=( Ian Abbott <abbotti@mev.co.uk> || MEV Ltd. is a company  )=-
-=( registered in England & Wales.  Regd. number: 02862268.  )=-
-=( Regd. addr.: S11 & 12 Building 67, Europa Business Park, )=-
-=( Bird Hall Lane, STOCKPORT, SK3 0XA, UK. || www.mev.co.uk )=-