Impossible to mount as tmpfs by default

[stronk7]

Thanks to #172 is now possible mount (bind) straight to XE or XE/XEPDB1. And it's working ok, so far.

But, for some reason, it seems that it continues being impossible to mount (tmpfs) the very same directories. The unzip execution fails with permissions error.

To reproduce it, just try running the container with:

--mount type=tmpfs,destination=/opt/oracle/oradata/XE (or XE/XEPDB1)

And you will get the zip errors.

Locally, I've workaround it by tmpfs mounting another (XE/XEPDB2) directory and then, in the entrypoint init sql scripts, cloning the whole XEPDB1 into XEPDB2. But it's not elegant and a waste of time.

So, finally, what we have done (and seems to be working) is to just ensure the permissions always (because the tmpfs mount is performed as root). So we have done (apart from other changes) this (just in case it can be a viable solution for the upstream images):

moodlehq/moodle-db-oracle@15eba81#diff-dd2c0eb6ea5cfc6c4bd4eac30934e2d5746747af48fef6da689e85b752f39557R4

With that we are able to tmpfs mount both XE or XE/XEPDB1 (and we are getting ~30 minutes quicker executions, from 4h40m down to 4h10m, aprox).

Ciao :-)

[gvenzl]

Hey @stronk7,

Thanks a lot for this information, however, I cannot make out what change solved the permission issues with tmpfs. The link you shared points to a commit with some async I/O functionality turned off, was that what was required to make tmpfs work?

[stronk7]

Hi @gvenzl,

so far, the only relaed change in that commit (that applies other, unrelated) was the linked line:

+ RUN chmod 777 -R /opt/oracle/oradata

With that, the tmpfs mount worked and the install/unzip doesn't fail with permissions errors. BTW, we have applied exactly the same chmod above to our oracle-free:23-slim image.

The rest of changes are unrelated, Moodle-specific or the disabling of async IO, that we are applying since 11g.

Ciao :-)

[gvenzl]

Hey @stronk7,

Ah I see, thanks a lot!
Yeah, I think I can add that to the upstream images, let me just think about a couple of more scenarios and whether this could cause any issues.

Thanks a lot!

[stronk7]

Just for the records, yesterday I went to rebuild our images (they were ~4 months old, from whenever I created this issue), and the new images are failing on startup with error:

CONTAINER: starting up...
CONTAINER: first database startup, initializing...
CONTAINER: uncompressing database data files, please wait...
ERROR: Cannot set file attribute : errno=1 : Operation not permitted : /opt/oracle/oradata/XE

And I've checked that disabling our attempt to mount with tmpfs shared in the issue description:

--mount type=tmpfs,destination=/opt/oracle/oradata/XE (or XE/XEPDB1)

Is what causes it (the container starts perfectly without it).

I've not had a chance to look to the changes since 4 months ago (they aren't many), will share here if I find something. Just wanted to warn in advance, that the proposed change of permissions is, now, not allowing the tmpfs mount to succeed.

Ciao :-)

[stronk7]

Just to confirm that it seems that mounting on tmpfs stopped working for us with #207 (was working before it). So, maybe, there is some operation that the:

7zzs x "${ORACLE_BASE}"/"${ORACLE_SID}".7z -o"${ORACLE_BASE}"/oradata/ > /dev/null

command is trying to perform, and cannot in a tmpfs mounted system... will try to debug it locally, I don't know much about 7z...

Ciao :-)

[stronk7]

Ok,

so the problem seems to be that we are mounting via tmpfs the /opt/oracle/oradata/XE directory, and later, when 7z tries to uncompress the files there it finds the directory existing, owned by root, and fails.

I've tried here mounting via tmpfs the whole /opt/oracle/oradata directory and then in works perfectly.

It would be awesome if the 7z command has some flag to continue on existing directories...

So, TLDR:

1. The original issue here continues being valid and ensuring that the oradata dir has 777 perms enables tmpfs to work on it, AKA:

+ RUN chmod 777 -R /opt/oracle/oradata

2. Right now, because of 7z failing (maybe it tries to change owner or some other attribute if it finds that some directory already exists), instead of mounting the oradata/XE (or oradata/XEPDB1) directories, the whole oradata needs to be mounted. This problem did not happen with unzip.

I'll check if there is any way to allow the 7z command to continue on existing directories...

Mystery solved, ciao :-)

Edited: I've been unable to find any switch of 7z able to disable the (timemodified or whatever) attributes updating that is causing the problem with the, root's owned, oradata/XE mount point... so only solution is, our side, to mount the whole oradata directory instead (too much, IMO, but our current CI infrastructure should support that without problem).

[gvenzl]

Hey @stronk7,

Thanks a lot for this comprehensive and most welcome detailed analysis!

I'll add the permission changes to the images so that you won't have to do that anymore.
Thanks for checking on 7z, we introduced it because it offers a better compression ratio and better uncompress times compared to zip.
I will also check and try to find out what 7z is doing with the parent directory – I think it's just trying to update the last modified date – and whether it can be somehow prevented.

However, thinking about this a bit more, mounting sub-directories only without the corresponding $ORACLE_BASE/oradata/dbconfig folder might lead to undesired results, such as the spfile not being reset on subsequent starts or similar. So technically speaking, mounting $ORACLE_BASE/oradata as a whole would appear to be the right solution but then again, I know that that also means having enough memory available to fit all files into the tmpfs.

[stronk7]

Thanks @gvenzl,

yeah, I can imagine that mounting only xe/free or other portable sub-directories is not ideal, in this case the main point of this is to get some speed benefit on tests, hence they are 100% disposable.

And, yes, curious that 7z fails with that, I was looking for available options to see if there was anything available about skipping existing directories or ignore perms failures... but didn't find anything.

In any case, right now we are tmpfs mounting the whole oradata, and it's working ok (luckily we have enough RAM in our CI workers to do that without too much trouble).

Again, thanks. Ciao :-)

[gvenzl]

Hey @stronk7,

Great, that's great to hear! The new version of the images with the changes should go live later today.
I know you already worked around it yourself anyway, but just an FYI.

Btw, if you like, I'd be more than happy to highlight your project(s) on https://github.com/gvenzl/oci-oracle-xe#users-of-these-images, if not done so already.

And btw, btw, you may want to, or already have considered, upgrading to gvenzl/oracle-free, the latest and greatest version of the free Oracle Database.

[stronk7]

Ah, we already have a 23c image based on your gvenzl/oracle-free. We still use the 21c the most (our real requirement for Moodle is Oracle 19 right now), but the 23c is there, and works perfectly, so far.

Ciao :-)

[gvenzl]

That's great, thanks for letting me know, and glad to hear that it works just fine as well!

Unless there is anything else, are you OK with closing this issue as completed?

[stronk7]

Completed, thanks!